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EXECUTIVE  SUMMARY 

AFRL/RXM  ManTech,  the  Manufacturing  and  Technology  Division  of  the  Air  Force  Research 
Laboratory’s  (AFRL)  Materials  and  Manufacturing  Directorate,  is  charged  with  understanding  and 
facilitating  successful  new  key  technologies  for  the  United  States  Air  Force.  In  the  past,  this  mission  has 
primarily  focused  on  understanding  and  facilitating  the  science  behind  these  new  technologies  and  the 
manufacturing  process  required  for  stable  product  availability.  Success  is  being  re-defined  to  include  the 
science  and  technology  maturation  process  which  includes  the  manufacturing  capability  and  business 
viability  of  the  organizations  providing  these  specialty  products. 

A  process  named  the  Rapid  Industrial  Preparedness  Assessment  (RIPA)  has  been  created  to  aid 
AFRL/RXM  ManTech’ s  technology  evaluators  in  assessing  this  definition  of  success.  The  RIPA  has 
been  developed  specifically  for  this  application  by  the  Center  for  Executive  Education  at  the  University  of 
Tennessee,  Knoxville  at  the  request  of  the  acting  AFRL/RXM  ManTech  Division  Chief. 

About  the  RIPA  -  The  RIPA  process  is  designed  to  be  a  quick  preliminary  evaluation  of  a  company 
based  on  readily  available  information  from  public  sources,  a  general  presentation  by  management,  a  tour 
of  their  facilities  and  short,  often  impromptu  conversations  with  three  to  four  front-line  employees.  This 
information  is  gathered  and  entered  into  a  custom  spreadsheet  which  allows  multiple  persons  to 
independently  enter  their  scores  on  27  items.  These  scores  are  weighted  with  respect  to  nine  categories. 
The  evaluators  then  reach  consensus  on  these  nine  categories  that  represent  business  viability.  The  data  is 
finally  represented  with  a  plot  that  suggests  the  company’s  value  stream,  operational  excellence,  and 
financial  health. 

Project  Scope  -  The  scope  of  this  project  is  to  1)  validate  the  RIPA,  2)  explore  the  RIPA’s  potential  to 
suggest  areas  where  a  business  may  need  to  focus  to  improve  viability  and  3)  to  explore  the  interaction  of 
the  RIPA  tool  with  the  existing  Manufacturing  Readiness  Assessment  (MRA)/Technology  Readiness 
Assessment  (TRA)  processes  of  AFRL/RXM  ManTech.  The  data  and  reasoning  for  the  conclusions 
discussed  in  this  summary  are  found  in  the  body  of  this  report,  as  well  as  additional  conclusions  and 
findings.  Use  of  these  summary  conclusions  and/or  the  RIPA  instrument  without  reading  the  entire 
report  is  not  recommended. 

Conclusions  -  The  RIPA  tool  and  processes  are  valid  and  reliable  methods  for  accurately  assessing  the 
business  condition  and  viability  of  a  small  company  or  a  facility  within  a  larger  company.  Trained 
ManTech  personnel  are  able  to  obtain  evaluations  consistent  with  expert  observers.  Those  experts  are 
able  to  match  results  with  persons  having  years  of  experience  with  a  company  after  only  a  four  hour 
RIPA  evaluation.  After  conducting  the  RIPA  assessment,  the  results  for  a  given  company  can  be  used  to 
1)  establish  strengths  and  weaknesses;  2)  create  action  plans  for  improvement;  3)  compare  companies  as 
one  tool  among  many  in  a  selection  process,  and  4)  append  the  knowledge  based  on  Manufacturing 
Readiness  Level  (MRL)  /  Technology  Readiness  Level  (TRL)  assessments  to  develop  a  holistic  picture  of 
the  risk  associated  with  technology  transfer. 

Recommendations  -  Among  many  others  discussed  later  in  the  body  of  this  report  the  key 
recommendation  of  this  report  is  that  AFRL/RXM  ManTech  management  should  require  the  RIPA  tool  to 
be  used  consistently  when  1)  new  companies  are  visited  for  the  first  time  to  assess  potential  USAF 
acquisition  of  new  products  requiring  extensive  technology  transfer  and  2)  for  all  companies  with  on¬ 
going  USAF  acquisition  or  technology  programs. 

In  addition  to  the  AFRL/RXM  ManTech  specific  recommendation  above,  the  RIPA  tool  could  be  a  useful 
tool  for  managers  and  decision  makers  who  find  assessing  plants  a  part  of  their  job  responsibilities. 

Availability  —  Full  documentation  of  the  RIPA  process,  the  associated  spreadsheet,  and  report  on  its 
development  and  validation  can  be  found  at  http://bus.utk.edu/usaf/ripa.htm. 
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1.0  INTRODUCTION 


AFRL/RXM  ManTech  is  charged  with  understanding  and  facilitating  successful  new  key  technologies  for 
the  United  States  Air  Force.  In  the  past,  this  mission  has  primarily  focused  on  understanding  and 
facilitating  the  science  behind  these  new  technologies.  Success  is  being  redefined  to  include  the  science 
and  technology  maturation  process,  manufacturing  capability  and  business  viability  of  the  organizations 
providing  these  specialty  products. 

A  tool  has  been  prepared  to  aid  AFRL/RXM  ManTech’ s  technology  evaluators  in  assessing  this  success. 
This  tool  is  named  the  Rapid  Industrial  Preparedness  Assessment  (RIP A)  and  has  been  developed 
specifically  for  this  application  by  the  Center  for  Executive  Education  at  the  University  of  Tennessee, 
Knoxville. 

The  RIPA  was  first  developed  in  early  2005.  The  tool  was  part  of  a  three-week  course  of  study  designed 
to  provide  persons  having  a  technical  background  with  a  primer  on  basic  business  principles.  The 
introduced  topics  included  financial  evaluation,  marketing,  leadership,  supply  chain  and  Lean 
manufacturing.  The  RIPA  was  used  at  the  conclusion  of  the  course  as  a  platform  to  tie  all  of  these  topics 
together.  Local  companies  were  visited  and  assessed  during  week  three  to  provide  a  hands-on  experience 
for  practicing  assessment  and  calibration  of  the  assessors.  See  Appendix  B  for  the  details  of  the 
development  of  this  tool. 

The  third  week  of  the  course  was  conducted  for  the  second  time  from  July  30  through  August  3,  2007. 
During  this  week  of  the  course,  participants  were  introduced  to  the  RIPA  tool  and  companies  were  visited 
for  practice  and  recalibration  of  the  assessors. 

1.1  Project  Scope  and  Hypotheses 

The  purpose  of  this  project  is  to  1)  validate  the  RIPA  process  and  2)  explore  the  RIPA  process’  potential 
to  suggest  where  a  business  may  need  to  focus  to  improve  viability.  A  research  team  composed  of  Allen 
Panned,  Elaine  Seat,  and  Maria  Weese  performed  site  visits  to  exercise  the  RIPA  process.  The  results 
were  documented  and  analyzed  subsequent  to  each  site  visit. 

The  data  used  in  this  analysis  was  provided  from  the  Assessors  (those  coming  into  the  company  and 
performing  the  RIPA)  and  from  the  company’s  management  team  (persons  employed  by  the  company 
being  visited).  A  complete  description  of  the  methodology  is  provided  in  section  2. 

This  project  tested  the  following  hypotheses: 

•  Hypothesis  1 :  Outside  assessors  can  arrive  at  an  accurate  evaluation  of  a  company’s  viability 
using  the  RIPA  process  (Assessor  and  management  team  RIPA  input). 

•  Hypothesis  2:  The  RIPA  process  can  identify  strengths  and  weaknesses  for  a  company  (Assessor, 
management  team,  and  qualitative  input). 

•  Hypothesis  3:  Outside  assessors  and  the  company’s  management  will  arrive  at  similar 
conclusions  regarding  business  viability. 

•  Hypothesis  4:  The  RIPA  tool  will  provide  the  same  conclusion  that  management  interviews 
suggest  regarding  past  and  present  states  of  the  organization  (Qualitative  Study). 

•  Hypothesis  5:  The  RIPA  tool  is  a  valid  and  reliable  instrument. 

•  Hypothesis  6:  The  RIPA  tool  can  be  used  in  conjunction  with  MRAs  and  TRAs. 
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2.0  METHODOLOGY 


A  study  was  designed  to  gather  data  that  would  provide  insight  into  the  project’s  hypotheses.  This  study 
collected  quantitative  and  qualitative  data  for  a  variety  of  companies  and  gathered  raw  input  from 
assessors  and  the  management  team.  This  data  was  subsequently  used  to  evaluate  the  six  hypotheses. 

Six  companies  were  visited  by  the  University  of  Tennessee  RIPA  team1  (UT  team)  specifically  for  this 
study.  Two  of  the  companies  are  participants  in  AFRL/RXM  ManTech  programs.  The  remaining  four 
are  a  from  a  spectrum  of  facilities  that  includes  manufacturing  raw  materials  processing  plants  and  high- 
technology  electronics  manufacturing  and  design.  This  variety  of  business  products  and  services  was 
deliberately  chosen  to  test  the  application  range  of  the  RIPA  process. 

Five  companies  were  visited  in  the  RIPA  refresher  training.  The  data  from  these  visits  are  also 
considered.  These  also  span  a  wide  variety  of  industries  and  company  size. 

2.1  RIPA  Data  Collection 

The  RIPA  instrument  is  based  on  input  from  evaluators  who  independently  rate  a  company  on  27  targeted 
items.  These  27  items  are  weighted  and  provide  scores  on  nine  categories  of  company  performance.  The 
evaluators  use  these  scores  to  subsequently  arrive  at  a  final  consensus  score  through  dialogue  among  the 
team  members.  This  final  consensus  score  is  based  on  the  evaluators’  personal  information,  observation, 
and  the  score  suggested  from  the  27  items.  Thus,  two  sets  of  data  for  the  nine  categories  are  available: 
the  27  item  suggested  score  and  the  nine  category  consensus  score.  Appendix  B  provides  a  detailed 
description  of  the  RIPA  instrument  and  interpretation  of  the  graphical  results. 

The  UT  team  arrived  at  their  results  by  each  person  independently  answering  the  27  item  set  and  then 
reaching  consensus  as  a  group  on  the  final  score  for  each  of  the  nine  categories.  This  is  the  intended 
mechanism  for  any  team  using  the  RIPA  instrument. 

The  data  gathering  process  followed  this  procedure: 

•  The  prospective  company  was  contacted  and  arrangements  were  made  for  a  visit.  They  were 
directed  to  treat  the  visit  as  if  the  UT  team  was  a  prospective  client  visiting  for  the  first  time. 

They  were  asked  to  provide  a  short  overview,  a  plant/facility  tour,  an  opportunity  for  a  short 
dialogue  with  their  management  team  and  brief  one-on-one  visits  with  a  few  shop  floor  workers 
and  supervisors. 

•  Before  the  visit,  the  project  team  did  an  internet  search  for  any  news  about  the  company  and  the 
company’s  market  sector.  Yahoo’s  financial  page  (free  service)  was  viewed  for  financial 
information.  In  the  case  of  private  companies,  other  public  companies  in  the  sector  were 
researched  to  arrive  at  assumptions  of  financial  information. 

•  A  half-day  site  visit  was  made  to  the  company. 

•  A  presentation  was  made  by  company  management  regarding  their  background,  marketing 
strategy,  products  and  future  plans.  This  presentation  was  not  prepared  specially  for  this  visit. 
Typically,  it  was  one  that  had  been  used  with  other  potential  clients.  We  specifically  requested 
that  the  participating  companies  NOT  prepare  a  unique  presentation  for  us,  but  that  they  use  one 
they  had  recently  used  with  another  prospective  client. 

•  The  project  team  provided  information  regarding  the  purpose  of  this  research. 


1  The  UT  Team  is  the  research  team  composed  of  Allen  Pannell,  Elaine  Seat,  and  Maria  Weese.  All  or  a 
combination  of  these  researchers  visited  each  site  and  performed  a  RIPA. 
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•  A  tour  was  made  through  the  facility. 

•  One-on-one  visits  were  conducted  with  employees  either  meeting  with  the  UT  team  privately  for 
a  short  discussion  or  in  impromptu  chats  while  on  the  facility  tour. 

•  The  management  team  was  asked  to  fill  out  a  set  of  questionnaires  (see  Table  1).  Data  indexes  6, 
7,  8  and  9  are  data  from  forms  the  management  team  filled  out  for  subsequent  input  into  a  RIPA 
tool  analysis). 

•  The  management  team  was  interviewed  to  gather  qualitative  data. 

•  The  project  team  completed  the  RIPA  tool  and  documented  their  visit. 

Gathering  data  for  this  study  required  choosing  companies  to  visit  and  executing  the  RIPA  process.  In 
addition,  other  data  were  collected  that  was  used  to  validate  the  process.  Once  the  data  set  was  gathered, 
it  was  used  in  multiple  analyses  to  test  the  hypotheses. 

The  type  of  data  gathered  and  its  relation  to  the  first  five  hypotheses  is  described  in  Table  1. 


Table  1.  Data  Gathered  and  Relationship  to  Hypotheses 


Hypothesis 

Data 

Index 

Description 

1 

2 

3 

4 

5 

1 

Company  Demographics 

2 

Qualitative  Interviews 

X 

X 

3 

Assessor’s  responses  to  27  items 

X 

X 

4 

Assessor’s  calculated  9  category  scores 

X 

5 

Assessor’s  consensus  scores  for  9  categories 

X 

X 

X 

X 

6 

Average  of  Management’ s  responses  to  27  items  for  the  present  day 

X 

X 

X 

7 

Average  of  Management’ s  responses  to  9  items  for  the  present  day 

X 

X 

X 

X 

X 

8 

Average  of  Management’ s  responses  to  27  items  for  5  years  ago 

X 

X 

X 

9 

Average  of  Management’ s  responses  to  9  items  for  5  years  ago 

X 

X 

X 

10 

Calculated  management  9  category  score  for  the  present  day 

X 

X 

X 

X 

X 

11 

Calculated  management  9  category  score  for  5  years  ago 

X 

X 

X 

Note  1:  Data  indices  10  and  1 1  are  the  nine  item  scores  as  calculated  from  the  27  item  inputs  in  data  indexes  6  and  8 
Note  2:  Additional  data  for  hypothesis  6  came  from  MRA/TRA  data  provided  by  AFRL/RXM  ManTech. 

2.2  Qualitative  Data  Gathering 

At  the  conclusion  of  the  site  visit,  the  management  team  at  each  company  was  asked  two  open-ended 
questions.  The  purpose  of  these  questions  was  to  provide  an  open  forum  for  information  that  may  not 
have  been  specifically  addressed  during  the  site  visit.  These  questions  were: 

•  What  is  different  from  five  years  ago  and  now? 

•  What  are  your  biggest  challenges? 
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2.3  Hypothesis  1  Research  Design 

Hypothesis  1  was  tested  by  a  comparison  of  data  indices  5,  7  and  10.  These  scores  were  analyzed  in  all 
categories  to  determine  if  there  were  statistically  significant  differences  in  scores  from  the  research  team, 
manager’s  direct  scoring  of  the  nine  categories  and  the  calculated  nine  category  scores. 

2.4  Hypothesis  2  Research  Design 

Hypothesis  2  was  tested  by  an  examination  of  the  nine  category  responses  of  each  company  to  identify 
areas  of  strength  and  weakness  for  that  company.  This  analysis  compared  the  researcher’s  identified 
areas  to  those  identified  by  the  management  team  in  their  scoring  and  qualitative  interviews. 

2.5  Hypothesis  3  Research  Design 

Hypothesis  3  was  tested  by  a  comparison  of  the  nine  categories  across  all  participating  companies.  The 
statistical  analysis  was  compared  with  the  qualitative  data  to  confirm  that  the  data  represented  what  was 
told  by  the  management  team. 

A  second  analysis  compared  the  management  team’s  scores  regarding  performance  five  years  ago  and 
their  present  day  scores  with  their  qualitative  descriptions  of  what  had  changed. 

2.6  Hypothesis  4  Research  Design 

Hypothesis  4  was  tested  by  comparing  the  nine  category  scores  for  the  present  and  five  years  ago  with  the 
qualitative  data  provided  by  the  management  team. 

2.7  Hypothesis  5  Research  Design 

Hypothesis  5  states  that  the  RIPA  is  a  valid  and  reliable  instrument.  Validity  and  reliability  were 
established  by  1)  evaluating  face  validity  of  the  27  items  and  2)  evaluating  significance  in  calculated 
versus  consensus  scores  for  the  nine  categories. 

2.8  Hypothesis  6  Research  Design 

Hypothesis  6  was  primarily  accomplished  through  the  evaluation  of  AFRL/RXM  ManTech 
documentation  on  the  two  visits  where  the  MRA/TRA  process  had  been  implemented. 
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3.0  RESULTS 


The  demographics  of  the  companies  visited  by  the  UT  team  are  provided  in  Table  2.  These  companies 
provide  a  broad  range  for  testing  the  RIPA  instrument.  Detailed  write-ups  for  each  company  can  be 
found  in  Appendices  E-J. 


Table  2.  Demographics  of  Participating  Companies 


Market  Distribution 

Company 

# 

Employees 

$  Annual 

Sales  (M) 

Sales  per 
employee 

Gov’t 

Commercial 

Global 

Presence 

1 

75 

$8 

$102,667 

20% 

80% 

Y 

2 

290 

$69 

$237,931 

75% 

25% 

Y 

3 

56 

$70 

$1,250,000 

0% 

100% 

Y 

4 

61 

$45 

$737,705 

0% 

100% 

Y 

5 

30 

$10 

$333,333 

100% 

0% 

N 

6 

75 

$13 

$173,333 

100% 

0% 

N 

3.1  Face  Validity 

A  key  aspect  in  analyzing  any  measurement  or  assessment  instrument  is  face  validity.  That  concept  is 
simply,  at  face  value,  do  experts  consider  the  instrument  does  what  it  is  intended  to  do.  For  the  RIPA 
instrument,  five  experts  where  surveyed  to  assess  face  validity.  The  experts  were: 

•  Ed  Miles,  President  Miles  and  Associates 

•  Dr.  Ken  Kirby,  Professor  Emeritus,  University  of  Tennessee 

•  Dr.  Ken  Gilbert,  Professor  and  Department  Chair,  University  of  Tennessee 

•  Dr.  Mandyam  Srinivasan,  Professor,  University  of  Tennessee 

•  Dr.  Dan  Flint,  Professor,  University  of  Tennessee 

Srinivasan  and  Flint  were  contributors  to  the  development  of  the  instrument.  They  were  asked  to  evaluate 
the  items  because  of  their  knowledge  of  the  purpose  and  intent  of  the  instrument,  in  addition  to  their 
extraordinary  expertise.  Miles,  Kirby  and  Gilbert  were  chosen  as  experts  not  involved  with  the 
development  of  the  instrument,  and  because  of  their  extraordinary  expertise  with  company  processes, 
organizational  excellence,  Lean  manufacturing  principles  and  management  science. 

A  survey  with  instructions  (see  sample  in  Table  3)  was  sent  to  each  expert,  in  addition  to  verbal 
instructions.  Each  expert  completed  the  survey  and  the  results  were  analyzed  by  the  authors  of  this 
report. 

3.2  Changes  made  based  on  the  survey 

While  several  new  questions  were  suggested  by  the  experts,  none  were  selected  as  additional  questions 
for  three  reasons: 
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•  No  suggested  questions  were  determined  to  be  superior  to  the  existing  questions. 

•  A  change  in  just  one  question  would  theoretically  require  a  new  validation  process. 

•  The  questions  as  currently  stated  have  proven  to  provide  a  valid  assessment. 

Based  on  the  ratings  of  the  questions  by  the  experts,  the  weighting  factors  of  six  questions  were  changed 
relative  to  the  scoring  of  the  RIPA.  These  changes,  while  making  common  sense,  did  not  make  a 
statistical  change  to  the  values  for  eight  of  the  nine  categories.  The  change  did  increase  the  Customer 
Satisfaction  category  average  by  0.1  in  the  direction  needed  to  correct  a  bias  that  was  noted  in  the 
validation  process. 


Table  3.  Sample  From  Face  Validity  Survey 


Market  Analysis 

No 

Minimal 

Some 

Strong 

Very 

Current  and  new  products  meet  and 
anticipate  the  needs  of  the  market  place 

Relation 

Relation 

What 

Relation 

Strong 

A  written  Marketing  Plan  is  being 

3 

4 

implemented 

1 

2 

2 

2 

5 

The  company  has  the  ability  to  access  the 

4 

5 

market  of  the  proposed  technology 

1 

2 

3 

2 

2 

The  proposed  technology  fits  with  the 

1 

4 

5 

current  strategic  plan  of  the  organization 

1 

2 

3 

2 

1 

Data  supports  that  a  market  exists  for  the 

4 

5 

proposed  technology 

1 

2 

3 

2 

2 

The  company  has  a  focused  Marketing 

4 

5 

Plan 

1 

2 

3 

2 

2 

Note:  The  survey  rating  scale  of  1  (No  Relation)  to  5  (Very  Strong)  is  shown  in  black  with  number  of  expert 
responses  regarding  the  strength  of  item  to  the  category  in  Red/italics  below  that  rating  number.  For  example,  the 
item  The  company  has  a  focused  Marketing  Plan  received  two  responses  for  Strong  Relation  and  two  responses  for 
Very  Strong  in  the  Market  Analysis  category. 


3.3  Reliability  based  on  Calculated  and  Consensus  Score  Differences 

Reliability  of  the  RIPA  tool  has  been  evaluated  by  analyzing  calculated  scores  versus  consensus  and 
specifically  input  scores. 

The  first  analysis  of  reliability  evaluated  the  differences  between  the  calculated  and  consensus  nine  item 
category  scores  of  the  research  team. 

The  second  analysis  of  reliability  evaluated  scores  in  data  indices  7  and  1 0  and  data  indices  9  and  1 1  for 
significant  differences  in  the  nine  category  scores  as  calculated  from  the  27  items  and  those  directly 
provided  by  the  management  teams. 
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4.0  DISCUSSION 


All  data  analysis  in  this  project  must  be  considered  qualitative.  Although  the  RIPA  generates  numerical 
values  and  graphs,  all  conclusions,  whether  reported  with  numerical  analysis,  graphs  or  otherwise,  must 
be  considered  to  have  come  from  a  qualitative  analysis  due  to  the  small  sample  size  and  complexity  of 
variables. 

Thus,  the  primary  limitation  of  this  study  is  the  inability  to  put  probabilities  on  statements  of 
conclusions. 

Hypothesis  1 :  Outside  assessors  can  arrive  at  an  accurate  evaluation  of  a  company ’s  viability  using  the 
RIPA  process.  (Assessor  and  management  team  RIPA  input) 

This  hypothesis  has  been  shown  to  be  true.  Demonstration  of  this  hypothesis  was  seen  in  three  different 
methods: 

•  The  success  of  UT  team  (in  a  four  hour  visit)  to  match  an  expert  analysis  of  two  companies  where 
the  expert  has  over  five  years  experience  with  each  company  and  access  to  all  relevant  data. 

•  The  success  of  UT  team’s  ability  to  mimic  management  assessments  during  the  validation  site 
visits. 

•  The  success  of  AFRL/RXM  ManTech  personnel  being  able  to  assess  companies  during  the  RIPA 
training  courses. 

•  Variability  is  demonstrated  across  all  of  the  companies  visited  and  accurately  portrayed  by  the 
UT  team,  AFRL/RXM  ManTech  teams,  and  the  management  teams. 

In  Method  a,  facilities  were  visited  where  one  of  the  UT  evaluators  had  extensive  experience  with  the 
plant.  Through  prior  work  at  the  corporate  office,  initial  assessments  over  the  last  five  years  of  the 
facilities,  personal  knowledge  of  the  plant  leadership  and  access  to  financial  and  other  relevant  data,  the 
expert  UT  evaluator  could  accurately  assess  the  facility  using  the  RIPA  tool.  This  evaluator  refrained 
from  interacting  in  the  rating  of  the  facilities  by  the  other  two  UT  evaluators  during  the  four  hour  standard 
RIPA  visit  and  research. 

Method  a  provides  the  most  striking  evidence  for  substantiation  of  hypothesis  1.  The  graphs  provided  in 
Figure  18  and  Figure  21  summarize  the  data  that  support  this  hypothesis.  In  both  of  these  cases,  Figure 
18  and  Figure  21  illustrate  the  ability  of  the  evaluators  to  match  almost  exactly  the  expert’s  analysis,  even 
though  the  perception  of  management  at  each  location  about  their  own  situation  was  quite  varied.  In 
essence,  the  RIPA  tool  in  capable  hands  with  the  standard  agenda  was  able  to  replicate  more  than  five 
years  of  intimate  knowledge. 

Method  b  is  a  more  approximate  measure  of  the  RIPA  process’  accuracy.  In  each  case  of  the  site  visits, 
management  was  asked  to  rate  their  company  on  the  27  questions  and  their  scores  were  averaged  to  create 
a  final  score.  Note  that  management  did  not  take  the  time  to  reach  consensus  on  their  answers.  While  the 
UT  team  believe  that  management  answers  would  have  some  positive  bias  in  general,  it  makes  sense  that 
no  one  would  have  a  better  understanding  of  a  company  than  senior  management  themselves.  Also  it  is 
worth  noting  that  some  of  the  information  the  evaluators  used  in  making  their  ratings,  were  of  course 
provided  by  management.  Every  effort  is  made  to  validate  the  information  through  observation,  internet 
research  and  employee  interviews.  The  expectation  was  that  the  evaluators  would  have  answers  close  to 
management  with  variation  expected  due  to  potential  management  bias  and  evaluator  lack  of  information. 
In  all  six  companies,  the  UT  team  and  management  ratings  were  similar  enough  to  suggest  the  RIPA 
process  to  be  a  valid  instrument.  In  five  of  the  six  cases,  management  had  ratings  on  the  RIPA  scale  that 
were  more  favorable  than  those  of  the  evaluators,  perhaps  an  indication  of  management  bias.  The 
individual  bubble  graphs  and  bar  charts  for  each  company  are  provided  in  the  appendices.  Due  to  the 


sample  size  being  small  in  statistical  terms,  comparisons  are  made  graphically  rather  than  numerically  in 
these  cases. 

Method  c  uses  the  RIPA  assessments  from  the  third  AFRL/RXM  ManTech  training  class  to  evaluate  the 
ability  of  the  typical  end  user  to  conduct  accurate  assessments. 

Method  c  has  been  conducted  three  different  times  since  the  creation  of  the  RIPA  instrument.  The  first 
two  times  were  associated  with  AFRL/RXM  ManTech  training  and  were  prior  to  the  commission  of  the 
research  effort.  The  third  effort  was  a  review  session  for  many  who  had  been  to  the  training  and  a  few 
attendees  that  had  not.  The  class  was  conducted  during  the  research  project  and  the  class  data  included  in 
this  study.  While  the  results  of  the  other  two  efforts  are  not  included  in  this  study  directly,  they  were 
consistent  with  those  discussed  here.  AFRL/RXM  ManTech  personnel  that  attended  the  one  week 
assessment  practice  event  were  placed  on  teams  for  each  of  five  different  plant  tours  and  assessments. 

The  teams  were  generally  assigned  at  random  and  rotated  on  a  daily  basis.  One  exception  occurred  when 
none  of  the  members  had  been  to  training.  (The  results  from  this  untrained  team  are  discussed  in  the 
recommendations . ) 

Figure  3  and  Figure  4  illustrate  the  results  for  the  class  visits  to  two  companies.  For  most  visits,  Figure  3 
is  indicative  of  the  results.  That  is,  the  teams  obtained  results  consistent  with  the  instructors  (experts). 
Again,  analysis  is  performed  graphically.  On  some  visits,  there  would  be  a  difference  of  opinion  by  one 
team  as  noted  in  Figure  4.  In  these  cases,  group  discussion  of  the  differences  would  serve  as  continued 
calibration  and  training  for  the  ManTech  personnel. 

While  on  some  visits  class  members  would  debate  the  exact  ratings  of  each  company,  the  final  graph  and 
relative  position  of  each  company  to  the  other  was  well  supported  by  all  student  groups. 

Method  d  places  the  results  for  all  of  the  companies  on  a  single  chart  for  the  management  team  and  for  the 
UT  team. 

Figure  1,  Figure  2,  and  Figure  5  illustrate  the  RIPA  process’s  capability  to  identify  the  variation  in 
companies  visited.  This  identification  of  variation  in  companies  visited  also  serves  to  support  the  ability 
of  the  process  to  distinguish  one  company  from  another. 
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Experts  Now 


Figure  1.  UT  Team  Assessments  for  All  Companies 


Management  5  Years  Ago 


Figure  2.  Management  Team  Five  Years  Ago  -  All  Companies 
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Conclusion  for  Hypothesis  1 :  The  data  support  the  hypothesis  and  it  is  the  opinion  of  the  report  writers 
that  the  RIPA  instrument  and  process  can  and  does  result  in  an  accurate  assessment  of  a  companies 
current  business  strengths  and  weaknesses  with  respect  to  the  RIPA  model.  The  instrument  has  been 
successfully  used  in  a  variety  of  industries,  size  of  companies  and  with  teams  with  varying  skills, 
backgrounds  and  experiences. 

NOTE:  the  RIPA  process  could  not  likely  have  identified  solutions  to  any  deficiencies,  but  certainly  could 
identify  where  those  deficiencies  lie. 


Company  W 


Figure  3.  Consistency  of  Class  Assessments  for  Company  W 
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Company  V 


Figure  4.  Consistency  of  Class  Assessments  for  Company  V 


All 


Figure  5.  Variation  in  Companies  (All  Companies  Visited  by  Class) 
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Hypothesis  2:  The  RIPA  process  can  identify  strengths  and  weaknesses  for  a  company.  (Assessor, 
management  team,  and  qualitative  input.) 

This  hypothesis  has  been  shown  to  be  true.  As  seen  in  the  company  write-ups  (Appendices  E  through  J), 
the  management  and  UT  team  assessments  are  similar  as  the  bubbles  typically  fall  in  the  same  quadrants.2 
The  qualitative  comments  from  management  support  their  strengths.  In  particular,  the  nine  categories 
show  the  same  level  of  score  between  all  assessments. 

In  addition,  although  there  is  not  significant  difference  between  the  UT  and  management  teams  for  a 
particular  company,  there  is  difference  between  companies.  This  suggests  the  RIPA  process  is  capable  of 
identifying  differences  in  performance  among  the  nine  categories. 

Hypothesis  3:  Outside  assessors  and  the  company’s  management  will  arrive  at  similar  conclusions 
regarding  business  viability. 

This  hypothesis  has  been  shown  to  be  true.  From  comparison  of  the  UT  team  and  Management  team 
RIPA  evaluations,  there  is  not  significant  difference  between  the  scores  (See  Figure  15,  Figure  18,  Figure 
21,  Figure  24,  Figure  27,  and  Figure  30).  The  bubbles  are  generally  within  the  same  quadrant,  and  thus 
suggest  the  same  strengths  and  weaknesses. 

Hypothesis  4:  The  RIPA  tool  will  provide  the  same  conclusion  that  management  interviews  suggest 
regarding  past  and  present  states  of  the  organization  (qualitative  study) 

This  hypothesis  has  been  shown  to  be  true.  Management  comments  about  their  company’s  strengths  and 
weaknesses  five  years  ago  and  the  present  follow  the  RIPA  graphs.  For  instance,  company  C  and  D  both 
referred  to  improvements  in  using  data  to  drive  change  and  improved  management.  For  both  of  these 
companies,  the  trend  was  toward  improved  Operational  Excellence  and  Value  Stream  as  described  in  the 
management  comments.  (See  Figure  18  and  Figure  21) 

Hypothesis  5 :  The  RIPA  tool  is  a  valid  and  reliable  instrument. 

This  hypothesis  has  been  shown  to  be  true.  Demonstration  of  this  hypothesis  can  be  seen  in  three 
different  methods; 

•  The  analysis  of  the  accuracy  of  the  instrument  for  Hypothesis  1  is  also  supportive  of  this 
hypothesis.  That  is,  for  an  instrument  to  be  valid  and  reliable,  it  must  by  definition  be  accurate. 

It  must  also  prove  to  be  useable  across  differing  situations,  companies  and  users.  All  of  these 
attributes  are  shown  to  be  true  in  the  Hypothesis  1  discussion. 

•  The  face  validity  study.  See  the  face  validity  discussion  in  Section  3. 

•  For  an  instrument/process  to  be  reliable,  it  must  be  able  to  be  replicated  over  time.  For  example, 
one  company  was  visited  in  all  three  training  events,  including  the  one-week  program  done  in 
conjunction  with  this  validation  process.  The  same  UT  trainers  visited  this  company  all  three 
times,  while  each  time  most  of  the  students  were  different.  In  the  judgment  of  the  UT  experts,  no 
significant  change  occurred  during  the  two  year  time  frame  of  visiting  this  company.  Each  visit 
was  scored  independently  by  the  experts  (although  prior  knowledge  exists)  and  the  scores  of  the 
students  were  certainly  independent  since  new  students  were  involved  each  time.  Figure  6  below 
shows  how  consistent  the  application  of  the  instrument  was  from  class  to  class  and  for  the  experts 
each  time. 


2  See  Appendix  B  for  an  explanation  of  the  RIPA  assessment  process  and  the  interpretation  of  the 
graphical  bubble  charts. 
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Company  V 


Figure  6.  Comparison  of  Three  Visits  to  Same  Company  Over  18  Months 


Hypothesis  6:  The  RIPA  tool  can  be  used  in  conjunction  with  MRAs  and  TRAs. 

Hypothesis  6  is  validated  from  a  comparison  of  the  MRA/TRA  data  for  two  of  the  companies  with  the 
RIPA  assessments  for  those  companies.  All  inferences  from  the  RIPA/MRA/TRA  support  one  another 
without  contradiction. 

The  key  implications  of  this  hypothesis  are  highlighted  in  Section  5:  Recommendations  and  Conclusions 
under  the  “Where  the  RIPA  assessment  fits  in  the  evaluation  process  category.” 

Appendix  C  provides  a  comparison  of  the  RIPA  questions  with  the  MRL  questions. 
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5.0  RECOMMENDATIONS  AND  CONCLUSIONS 


The  RIPA  tool  proved  to  be  a  valid  and  reliable  instrument.  An  analysis  of  the  entire  collected  data  set 
shows  many  consistencies  that  suggest  the  instrument  is  a  reliable  general  measure  of  a  company’s 
viability.  Certainly,  any  business  transaction  must  be  based  on  full  due  diligence  and  detailed  analysis, 
but  the  RIPA  can  suggest  a  starting  place  for  evaluating  a  company  with  respect  to  overall  business 
viability. 

The  following  conclusions  and  recommendations  have  resulted  from  this  project. 

Consistent  application  of  the  RIPA  process  has  value: 

•  When  new  companies  are  visited  for  the  first  time  to  assess  potential  participation  in  USAF 
research,  development  and  acquisition  programs. 

•  With  all  companies  where  the  USAF  has  on-going  research,  development  and  acquisition 
programs. 

5.1  Training 

New  AFRL/RXM  ManTech  personnel  should  receive  training  on  the  RIPA  tool  before  participating  or 
leading  an  assessment  team.  This  training  would  most  likely  involve  a  subset  of  the  following,  depending 
on  the  person’s  background,  and  is  recommended  to  be  developed  using  the  knowledge  gained  by  this 
research  project  and  explained  below: 

•  With  proper  training,  AFRL/RXM  ManTech  personnel  successfully  used  the  RIPA.  From  the 
cross  section  of  backgrounds  in  the  training  program  that  were  supervised  during  the  field  work 
portion  of  the  training  course,  it  is  evident  that  persons  with  a  background  in  technology 
evaluation  can  learn  the  basics  listed  in  the  nine  categories  of  the  RIPA  adequately  to  use  the 
instrument. 

•  On-going  training  is  required  for  a  uniform  use  of  this  instrument.  During  week  three  of  the 
training  course,  AFRL/RXM  ManTech  personnel  that  had  been  trained  in  the  previous  year  made 
site  visits  in  teams  to  several  companies.  At  the  conclusion  of  the  visit,  each  team  completed  the 
RIPA  process  and  presented  their  results  to  the  entire  group.  There  was  substantial  difference 
between  the  team  evaluations  after  the  first  site  visit,  but  the  teams  became  more  calibrated  and 
consistent  with  each  visit  and  subsequent  discussion.  This  technique  of  assessment  and  then 
comparison  of  observation  and  rating  is  an  established  technique  for  training  and  calibrating 
expert  observers. 

•  All  program  managers  are  recommended  to  participate  in  at  least  two  RIPA  events  annually  to 
maintain  calibration  and  expertise.  Mangers  could  participate  on  teams  doing  live  assessments 
and/or  practice  assessments  could  be  arranged  as  they  were  during  this  research  project. 

•  The  third  week  of  the  training  course  was  offered  in  2006  and  then  a  refresher  course  in  2007.  In 
the  2007  class,  there  were  participants  who  had  not  been  to  week  one  and  two,  but  were  invited  to 
participate  in  the  field  work.  These  participants  had  similar  backgrounds  to  the  trained 
participants,  but  were  initially  placed  in  a  team  together  so  that  the  research  team  could  evaluate 
the  value  of  having  the  full  training  course.  The  RIPA  results  from  this  team  were  significantly 
skewed  from  the  teams  of  experts  and  trained  participants  (See  Figure  9).  Thus,  training  in  all 
nine  aspects  of  business  viability  and  calibration  of  evaluation  with  field  work  is  required  for 
accurate  and  calibrated  performance  of  a  RIPA. 
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Training  for  conducting  consistent  RIPA  requires: 

•  Read  appropriate  sections  in  the  RIPA  training  manuals. 

•  Attend  MRA/TRA  training. 

•  Serve  as  an  observer  on  at  least  one  RIPA. 

•  Attend  public  seminars  on  specific  topics  related  to  RIPA. 

•  Participate  as  a  RIPA  team  member  on  the  first  live  assessment  rather  than  as  a  leader. 

5.2  Application  of  the  RIPA 

Personal  impacts  on  accurate  application  (personal  bias)  were  observed. 

•  Bias  was  observed  in  the  assessment  of  two  Companies.  On  two  site  visits,  the  UT  team  was 
accompanied  by  persons  closely  associated  with  the  companies  being  evaluated.  The  UT  team 
and  these  persons  independently  completed  the  RIPA  (see  Figure  7  and  Figure  8  where  their 
circle  is  identified  as  the  Helpers).  In  both  cases,  bias  was  demonstrated  by  the  team  with  a 
personal  interest  in  the  success  of  the  company.  It  is  recommended  that  independent  observers 
conduct  a  RIPA  to  avoid  bias. 

•  Inter-views  with  a  variety  of  persons  in  the  company  is  crucial.  The  UT  team  discovered  that 
management  tends  to  be  more  positive  about  performance  than  perhaps  is  accurate.  In 
interviewing  a  range  of  persons,  management’s  views  were  sometimes  supported,  but  often  we 
learned  the  extent  to  which  the  workforce  was  onboard  with  the  implementation  of  lean,  issues 
about  workforce  quality,  and  confidence  in  the  company’s  ability  to  proceed  were  not  the  same. 
These  interviews,  coupled  with  the  tour,  provided  additional  information  that  was  helpful  in 
reaching  consensus  scores. 

Where  the  RIPA  fits  in  the  evaluation  process: 

•  Companies  can  be  compared  via  the  RIPA.  (reference  individual  company  write-ups  in 
Appendices  E  through  J).  The  companies  visited  in  the  training  class  provided  a  range  of 
capabilities  and  maturity  in  their  business  processes.  These  differences  and  variety  were  apparent 
in  the  RIPA  graph  of  all  companies  plotted  on  a  single  chart  (see  Figure  10).  This  variety 
demonstrates  the  instrument’s  ability  to  discern  differences  in  company  viability. 

•  The  RIPA  tool  can  not  be  used  to  determine  whether  or  not  to  do  business  with  a  company.  The 
RIPA  merely  suggests  areas  that  the  company  may  need  to  focus  resources  to  improve  business 
viability.  Due  diligence  and  further  exploration  is  always  required  to  determine  the  exact  nature 
of  strengths  and  weaknesses. 

•  The  RIPA  tool  can  be  used  in  conjunction  with  MRAs  and  TRAs.  The  RIPA  provides  additional 
information  at  the  company  level  regarding  strengths  and  weaknesses.  The  RIPA  provided 
information  on  variables  independent  from  those  in  the  MRAs  and  TRAs.  The  RIPA 
complements  the  specific,  detailed  manufacturing  and  technology  assessment  by  providing 
information  about  the  company’s  viability  in  providing  a  business  environment  for  success  of  the 
specific  product. 

5.3  Miscellaneous  Observations 

•  In  the  case  of  observed,  significant  improvement  of  a  company  (both  self-reported  and  expert 
observation  in  the  case  of  Company  B  and  Company  C),  leadership  was  always  determined  to  be 
the  primary  factor  which  drove  improvement.  For  example,  Company  F  attributes  their 
improvement  to  strong  leadership  in  fiscal  matters,  and  Company  E  attributed  their  improvement 
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to  a  complete  change  of  management.  Thus,  company  leadership  must  be  strongly  considered  in 
predicting  the  ability  to  improve  viability. 

•  During  the  company  visits  made  by  the  UT  team,  management  provided  data  for  both  the  27 
items  and  the  nine  categories  for  their  company  for  both  the  present  state  and  five  years  ago. 
Statistically  significant  differences  were  found  between  the  final  results  from  these  two  methods, 
with  the  input  of  the  27  items  more  closely  aligning  with  the  evaluation  from  the  UT  team.  The 
UT  team  believes  that  the  nine  categories  are  too  general,  while  the  27  items  are  specific  and 
identify  behaviors.  This  specificity  leads  to  more  calibrated  results.  Thus,  the  RIP  A  can  not  be 
short-cut  to  just  answer  the  nine  categories. 

•  AFRL/RXM  ManTech  should  resist  the  natural  temptation  by  those  using  the  RIPA  to  make 
changes  and  improvements  to  the  RIPA  process.  The  process  and  instrument  have  now  been 
validated  and  shown  to  perform  well  at  its  intended  purpose.  Any  changes  will  likely  yield  only  a 
small  improvement  (if  any  at  all)  and  ultimately  lead  to  an  instrument  that  is  no  longer  valid  and 
likely  more  complex. 

•  The  RIPA  process  is  a  useful  tool  for  managers  and  decision  makers  whose  duties  include  the 
assessment  of  manufacturing  plants  to  accomplish  tasks  for  the  USAF. 

5.4  Future  Implications  and  Actions 

The  RIPA  has  proved  to  be  a  valuable  tool.  It  encourages  program  managers  who  are  considering 
company  capabilities  to  look  beyond  only  that  with  which  they  are  familiar.  The  tool,  when  used  in 
conjunction  with  the  detailed  assessments  of  manufacturing  and  technology  readiness,  can  provide 
insights  about  the  business  foundation  that  underpins  the  specific  program  of  interest  to  the  USAF. 

However,  gauging  the  RIPA’s  ability  to  predict  viability  over  time  will  be  difficult.  A  true  longitudinal 
study  would: 

•  Evaluate  a  company  now  and  predict  what  should  be  done  to  improve  viability. 

•  Evaluate  the  company  in  the  future  and  assess  the  extent  to  which  the  predicted  improvements 
were  carried  out  and  made  a  difference. 

This  technique  ignores  synergistic  effects,  market  impacts  and  the  myriad  of  forces  that  shape  a 
company’s  success. 

Another  study  to  evaluate  the  RIPA’s  effectiveness  would  randomly  choose  companies  to  have  the  RIPA 
performed.  The  results  would  then  be  used  by  the  company  in  future  planning  based  on  the  evaluation  of 
current  capabilities.  However,  it  is  unproductive  and  expensive  to  only  use  the  tool  on  specific 
companies  to  see  if  those  with  out  RIPA  information  are  less  successful.  Thus,  the  test  subjects  will 
never  be  available  for  a  statistical  study  of  effectiveness.  In  addition  for  those  companies  purposefully 
not  selected  for  a  RIPA,  not  providing  knowledge  gleaned  from  this  type  of  evaluation  in  the  name  of 
research  would  be  unethical. 

The  RIPA  effectiveness  is  recommended  to  be  measured  qualitatively  through  anecdotal  evidence 
provided  by  program  managers.  This  evaluation  will  be  dependent  on  the  RIPA  being  used  with  other 
assessment  tools  required  by  program  managers. 
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Company  G 


Figure  7.  Bias  in  Rating  at  Company  G 

Note:  The  Helpers  circle  is  the  team  of  untrained  participants.  The  other  circles  are  from  the  UT  team,  and  the 
company  management  team. 


Company  A 


Figure  8.  Bias  in  Rating  at  Company  A 

Note:  The  Helpers  circle  is  the  team  of  untrained  participants.  The  other  circles  are  from  the  UT  team,  and  the 
company  management  team. 
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Company  V 


Figure  9.  Evaluation  of  Company  V  by  Experts,  Trained  Participants,  and  Untrained  Participants 


All  Companies  from  Training  Class 


Operational  Excellence 


Figure  10.  Average  Score  From  Companies  Visited  in  Field  Training  2007 
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APPENDIX  A:  ABBREVIATIONS  AND  ACRONYMS 


AFRL/RXM  Air  Force  Research  Laboratory,  Materials  and  Manufacturing  Directorate 
BPA  Business  Practices  Assessment 

ManTech  Manufacturing  and  Technology  Division  of  AFRL/RXM 

MRA  Manufacturing  Readiness  Assessment 

MRL  Manufacturing  Readiness  Level 

MRO  Maintenance,  Repair,  and  Overhaul 

MRP  Material  Requirements  Planning 

NIST  National  Institute  of  Standards  and  Technology 

RIPA  Rapid  Industrial  Preparedness  Assessment 

RPA  Rapid  Plant  Assessment 

SYMLOG  Systematic  Multiple  Level  Observation  of  Groups 
TRA  Technology  Readiness  Assessment 

TRL  Technology  Readiness  Level 

USAF  United  States  Air  Force 

WIP  Work-in-Progress 
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APPENDIX  B:  THE  RIPA  ASSESSMENT  TOOL  AND  PROCESS 
The  Rapid  Industrial  Preparedness  Assessment  (RIPA) 

The  RIPA  is  a  process  for  assessing  business  viability.  It  is  intended  to  be  used  during  both  formal  and 
informal  site  visits  and  provides  a  format  for  observing  standard  practices  and  asking  targeted  questions 
about  all  areas  of  the  company. 

The  components  of  the  RIPA  are: 

•  Advance  preparation  through  an  internet  search  on  the  company,  news  about  the  company’s 
business  sector,  any  news  about  the  company  and  financial  information  commonly  available  from 
Yahoo  or  Google. 

•  A  site  visit  to  the  participating  company. 

•  General  information  about  the  company  (typically  from  a  generic  management  presentation). 

•  A  tour  through  the  facility. 

•  Casual  conversation  with  a  few  employees  from  across  the  company  about  how  it  is  to  work  there 
and  what  they  would  change. 

•  Use  of  the  RIPA  Excel  spreadsheet  to  establish  a  baseline  assessment  of  business  viability. 

The  RIPA  Tool 

The  RIPA  tool  is  an  excel  spreadsheet  with  a  customized  interface.  It  allows  the  user  to  enter  the 
company  name  and  evaluators’  names,  and  then  a  unique  spreadsheet  is  automatically  generated  to  be 
used  to  assess  the  company  on  that  visit. 

After  the  site  visit,  the  evaluators  independently  score  the  company  on  27  targeted  items.  These  items  are 
listed  in  Table  4.  These  27  items  provide  scores  on  the  nine  main  categories  listed  in  Table  5.  The  27 
items  are  weighted  to  have  a  primary,  secondary  or  tertiary  effect  on  the  nine  main  categories. 

The  nine  main  categories  are  grouped  into  three  independent  variables:  Financial,  Value  Stream,  and 
Operational  Excellence.  The  mapping  of  the  nine  categories  into  the  independent  variables  is  provided  in 
Table  6. 

After  the  evaluators  have  entered  their  independent  rating  on  each  of  the  27  items,  an  average  score  is 
calculated  for  each  of  the  nine  items  from  this  aggregated  input.  The  assessment  team  then  uses  this 
suggested  score  for  each  of  the  nine  categories  to  discuss  and  reach  a  consensus  score  for  the  individual 
cagegory.  The  team  is  encouraged  to  discuss  what  they  have  discovered  in  their  preliminary  preparation, 
observation  during  the  visit  and  any  other  knowledge  they  might  have  of  the  company  and  this  business 
sector  to  arrive  at  a  consensus  score. 
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Table  4.  RIPA  27  Items  for  Input  of  Independent  Assessment  by  Evaluators 


Item  No 

Targeted  Item 

1 

Workplace  design  supports  effective  process  flow 

2 

A  written  Marketing  Plan  is  being  implemented 

3 

Constraints  are  being  managed  in  key  processes 

4 

Customer  contact  employees  arc  empowered  to  resolve  problems  appropriately 

5 

Work  Cells  are  utilized  where  appropriate 

6 

Customer  satisfaction  data  guides  the  quality  improvement  efforts 

7 

Customer  satisfaction  measures  are  well  communicated  across  organization. 

8 

Employees  arc  empowered  to  take  necessary  action  to  improve  processes 

9 

Evidence  exists  that  the  company  has  or  could  obtain  adequate  capital  when  necessary 

10 

Financial  performance  exceeds  that  of  competitors 

11 

Established  vision,  mission,  values,  goals,  and  objectives  are  understood  at  all  levels 

12 

Key  processes  arc  being  continuously  improved  to  meet  stakeholders  needs 

13 

Leadership  behavior  is  consistent  with  stated  values 

14 

Partner  relationships  are  built  with  key  suppliers  and  customers 

15 

Process  cycle  time  is  being  minimized  in  key  business  processes 

16 

Relevant  information  is  openly  shared  with  key  suppliers 

17 

Safety  is  an  obvious  priority  for  the  organization 

18 

Simple  statistical  tools  are  used  effectively  across  the  organization  to  achieve  performance 
improvement. 

19 

The  company  has  adequate  Cash  flow  to  sustain  operations  for  at  least  one  year 

20 

The  company  has  the  ability  to  access  the  market  of  the  proposed  technology 

21 

The  entire  Value  Chain  is  managed  as  a  key  process 

22 

The  organization  systematically  benchmarks  against  its  competition  (or  against  best  in  class) 

23 

The  proposed  technology  fits  with  the  current  strategic  plan  of  the  organization 

24 

The  use  of  Lean  tools  is  extensive 

25 

Trained  internal  continuous  improvement  champions  are  working  in  the  organization 

26 

Data  supports  that  a  market  exists  for  the  proposed  technology 

27 

A  Problem  Solving  methodology  is  actively  used  by  all  employees 
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Table  5.  RIPA  Nine  Categories  of  Business  Viability 


Category 

Number 

Category  Title 

1 

Market  Analysis 

2 

Supply  Chain 

3 

Customer  Satisfaction 

4 

Leadership 

5 

Quality 

6 

Lean  Operations 

7 

Profit 

8 

Cash  Flow 

9 

Capital 

Table  6.  Mapping  of  Nine  Categories  into  Independent  Variables 


Independent  Variable 

Category 

Value  Chain 

Market  Analysis 

Supply  Chain 

Customer  Satisfaction 

Operational  Excellence 

Leadership 

Quality 

Lean  Operations 

Financial 

Profit 

Cash  Flow 

Capital 
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The  consensus  scores  of  the  three  categories  are  then  summed  to  reach  the  independent  variable  score. 
The  three  independent  variables  are  represented  on  a  graph,  where  financial  data  is  plotted  as  a  circle  at  x 
and  y  coordinates  where  Operational  Excellence  is  the  x-axis  and  Value  Stream  is  the  y-axis.  A  sample 
bubble  chart  is  provided  in  Figure  1 1 .  The  size  of  the  bubble  represents  the  financial  score.  The  larger 
the  bubble  the  greater  the  estimated  financial  viability. 


The  RIPA  bubble  chart  provides  a  graphical  representation  of  the  viability  of  a  company.  Scoring  for  all 
three  constructs  (Value  Chain,  Operational  Excellence,  and  Financial)  has  a  range  of  0  to  15.  The  circle 
suggests  the  financial  viability  with  the  benchmark  circle  simply  providing  a  reference  for  a  score  of  7.5. 
While  there  is  no  absolute  scoring  that  correlates  to  an  action,  the  relative  size  and  position  of  the  bubble 
suggests  a  company’s  viability  in  the  three  constructs. 
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Figure  11.  Sample  RIPA  Bubble  Chart 

Sample  Scenario: 

A  team  enters  their  individual  rating  for  the  27  items  into  the  RIPA  spreadsheet  for  Company  Q.  This 
input  generates  weighted  average  scores  as  shown.  During  the  tour,  it  appeared  that  the  company  has  a 
good  foundation  in  lean  operations,  ffowever,  one  of  the  team  members  has  followed  this  company  for 
an  extended  time,  and  knows  that  although  Lean  is  in  place  now,  it  is  a  new  way  of  thinking  for  this 
company.  The  team  decides  to  lower  the  scores  for  quality,  supply  chain,  and  Lean  based  on  their 
discussion  and  re-evaluation.  Table  7  provides  sample  data  for  this  scenario.  Figure  12  provides  the 
bubble  chart  for  this  scenario. 
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Table  7.  Sample  Scenario  of  Determining  Assessment  Scores 


Independent 

Variable 

Category 

Calculated 

Average 

Consensus 

Score 

Independent 

Variable 

Score 

Value  Chain 

10.45 

Market  Analysis 

3.2 

3.2 

Supply  Chain 

4.1 

3.75 

Customer  Satisfaction 

3.5 

3.5 

Operational 

Excellence 

10.0 

Leadership 

3.5 

3.5 

Quality 

4.2 

3.5 

Lean  Operations 

4.5 

3 

Financial 

11.0 

Profit 

4.2 

4.2 

Cash  Flow 

3.3 

3.3 

Capital 

3.5 

3.5 

Figure  12.  Bubble  Chart  of  Sample  Scenario 
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APPENDIX  C:  HISTORY  AND  BACKGROUND  OF  THE  PROJECT 


RIPA  Origins 

The  Rapid  Industrial  Plant  Assessment  was  developed  by  the  University  of  Tennessee  at  the  request  of 
Bill  Russell  -  Chief  of  the  Electronics  Branch  for  the  Air  Force  Research  Lab,  Materials  and 
Manufacturing  Directorate.  Its  original  intent  was  to  serve  as  a  tool  to  reinforce  training  of  program 
managers  in  the  areas  of  Finance,  Marketing  and  Operational  Excellence.  This  training  was  developed  to 
enhance  the  skills  of  technical  associates  in  areas  of  business  fundamentals.  The  main  goal  was  to  open 
their  eyes  to  issues  beyond  technical  ones  and  to  assist  in  evaluating  risk  related  to  technology  transfer 
from  the  Air  Force  to  the  private  sector.  Past  experience  had  shown  that  business  readiness  (or  lack 
thereof)  was  a  key  factor  in  unsuccessful  technology  transfers  as  much  as,  or  more  so  than  technical 
issues. 

The  original  instrument  was  fashioned  based  on  the  following  elements; 

•  The  Rapid  Plant  Assessment  (RPA)  utilized  in  the  University  of  Tennessee  Aerospace  MBA 
program. 

•  The  specifications  and  objectives  stated  for  the  initial  project. 

•  The  experience  of  the  University  of  Tennessee  staff  who  developed  it. 

•  The  Malcom  Baldridge  Award  criteria. 

•  SYMLOG,  a  survey  instrument  developed  for  team  building. 

Each  of  the  above  elements  are  discussed  and  referenced  below. 


Original  Specifications 

The  key  summary  of  objectives  is  as  follows,  based  on  the  actual  training: 

•  Utilize  the  RIPA  as  a  tool  to  understand  the  stability  of  the  company,  its  progress  with  lean,  and 
the  strengths  and  weaknesses  that  may  affect  the  probability  of  success  for  the  technology 
transition. 

•  Understand  how  to  determine  whether  a  market  exists  or  will  exist  for  a  particular  technology  or 
product  (specifically,  assess  whether  a  business  plan  exists  for  transitioning  a  new  technology 
into  an  existing  or  new  product). 

•  Understand  what  types  of  financial  information  must  be  gathered  to  ascertain  the  financial 
strength  of  an  organization. 

•  Accurately  interpret  the  financial  information  gathered  to  determine  an  organization's  financial 
stability. 

•  Understand  what  financial  resources  are  necessary  to  bring  a  new  technology  to  production. 

•  Better  understanding  of  the  implications  of  the  supply  chain  on  the  well  being  and  future  viability 
of  an  organization. 

•  Understand  how  Lean  and  Six  Sigma  impact  an  organization  and  an  organization's  progress  on 
the  Continuous  Improvement  journey  can  be  observed. 

•  Understand  what  type  of  assessment  team  should  be  assembled  to  do  the  RIPA,  financial  analysis, 
and  any  other  activities  deemed  necessary. 


26 


Rapid  Plant  Assessment 

The  Rapid  Plant  Assessment  (RPA)  was  designed  by  Gene  Goodsen  an  adjunct  professor  in  Operations 
Management  at  the  University  of  Michigan  to  assist  managers,  consultants  and  students  in  performing  a 
rapid  evaluation  of  the  state  of  their  operation.  The  purpose  of  the  RPA  is  to  get  an  accurate  snapshot  of  a 
company  from  a  short  (30  minute)  tour  (often  this  is  all  a  competitor  will  allow).  "With  an  educated  eye, 
one  could  discern  a  plant's  strengths  and  weaknesses  accurately"(Goodsen).  The  information  the  RPA 
has  provided  has  influenced  activities  and  decisions  ranging  from  benchmarking  to  competitor  analysis  to 
strategic  acquisitions.  The  RPA  results  of  a  tour  are  available  in  a  day  or  less  whereas  most  rating 
systems  would  take  weeks.  The  RPA  also  helps  to  judge  the  level  of  Lean  in  an  operation,  prioritize  the 
targets  of  opportunity  and  facilitate  improvements.  The  RPA  asses  a  plant  in  the  following  1 1  categories: 
Customer  satisfaction,  Safety  and  Environment,  Visual  Management  Deployment,  Scheduling  System, 
Product  Flow,  Inventory  and  WIP  Levels,  People  teamwork  and  Skill  Level,  Equipment  and  Tooling 
State,  Ability  to  Manage  Complexity  and  Variability,  Supply  Chain  Integration  and  Quality  System 
Deployment.  Each  of  these  categories  is  rated  on  a  1,  3,  5,  7,  9,  11  scale  where  1  is  Poor  and  1 1  is  Best  in 
Class  and  only  one  company  in  each  industry  should  receive  this  rating. 

•  Customer  Satisfaction  and  Customer  ratings,  quality  certifications  and  ratings,  warranty  and 
product  liability  costs,  employee  knowledge  of  external  and  internal  customer  requirements, 
visitor  materials  and  welcome,  market  share,  rate  of  new  product  introduction  and  acceptance. 

•  Safety,  Environment,  Cleanliness  and  Order:  Safety  and  environment  record,  place  for 
everything  and  everything  in  its  place,  cleanliness  of  operations,  exterior  and  interior  doors, 
equipment,  spills,  leaks,  noise,  lighting,  paint,  dust,  air  quality,  employee  dress,  restroom 
conditions,  desks  and  workbench  order  and  cleanliness  degree  of  spiffing  for  visitors  (negative), 
inventory  order,  material  flow  order  and  cleanliness,  color  and  other  coding  for  safety  and  order. 

•  Visual  Management  Deployment:  Operations  mission  and  performance  objectives  visible, 
visibility  of  labeling  and  coding  of  product  lines,  inventory,  equipment  and  tooling,  color  coding 
and  differentiation,  visibility  of  customer  identification  and  ratings,  visibility  of  charts  tracking 
operations  and  team  safety,  quality  and  productivity,  control  room  showing  status  of  total 
operation,  customer  order  and  order  fulfillment  visibility,  Kanban  deployment,  inventory  count 
can  be  made  visually,  machines  and  tool  labeling,  costs,  preventive  maintenance  visibility, 
product  displays,  audit  results  visible. 

•  Scheduling  System:  Degree  of  scheduling  to  customer  order,  order  process  efficiency,  product 
line  scheduling  at  single  point,  scheduling  buckets  (each  order,  hourly,  daily,  weekly,  monthly), 
supplier  scheduling  and  delivery,  replenishment  versus  order  fulfillment,  computer  scheduling 
versus  Kanban,  pull  versus  push  systems,  flow  time  efficiencies,  back  room  costs  of  scheduling, 
MRP  costs,  data  entry  costs. 

•  Product  and  Material  Flows:  Product  line  versus  shop  layout,  rolling  carts  pulled  by  tractors  or 
by  hand  or  conveyors  versus  forklifts,  travel  distances  between  processes,  material  movement 
responsibility  (process  owned  or  separate  material  staff,  container  one  size,  containers  designed 
for  parts  families,  single  versus  multiple  docks  to  minimize  material  travel,  space  utilization, 
goals  for  space  use  reduction). 

•  Inventory  and  WIP  Levels:  WIP  levels  at  each  process,  WIP  in  transit  in  plan,  separate  stores 
versus  line  side  storage,  number  of  inventory  storage  areas,  finished  product  levels,  total 
inventory  to  sales  ratio,  process  cycle  time  to  flow  time  ratios,  accountability  of  inventory,  WIP 
movement  triggered  by  computer,  material  department  or  next  process,  theoretical  versus  actual 
flow  times. 

•  People  teamwork,  skill  level:  Team  problem  solving  capability  and  history,  employee 
willingness  to  talk  about  customers,  products  and  company,  uniformity  of  dress,  communications 
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and  recognition  environment,  sales  per  employee,  team  meeting  areas  and  performance  charts, 
training  investments,  educational  support,  benefit  package  and  costs,  unionization  activity, 
workforce-management  relationship,  community  support,  company-supported  activities  (picnics, 
open  houses,  sports  teams),  local  involvement,  employee  knowledge  of  support  and  of  customers 
and  business,  work  instructions  standards. 

•  Equipment  and  Tooling  State:  Preventative  maintenance  system,  setup  change  times,  integrated 
go/no  go  quality  checks,  machine  performance  data  availability,  knowledge  and  utilization  of 
bottleneck  processes,  process  control  capability,  total  asset  utilization,  operator  routine 
maintenance,  maintenance  staff  and  teams,  MRO  replenishment  efficiency,  tool  and  fixture 
orderliness,  cleanliness  and  storage  location,  equipment  improvement  policy,  equipment 
technology  strategy. 

•  Ability  to  Manage  Complexity  and  Variability:  Use  of  common  parts,  processes,  and  procedures 
prototype  process,  paper  transactions  required  on  floor,  keyboard  entries  versus  bar  coding,  back 
room  paperwork  and  computer  transactions  costs,  matching  of  data  collected  with  data  needed, 
simplicity  and  clarity  of  operations  layout,  indirect  to  direct  labor  ratio,  support  staff  to  total 
workforce  ratio,  overhead  cost  ratios,  commonality  of  tooling  and  fixturing,  commonality  of 
equipment  and  tools,  commonality  of  support  software  and  applications  programs  across  the 
operation  among  sister  plants,  equipment  efficiencies,  ability  to  handle  variable  demand,  ability 
to  eliminate  controllable  variations,  ability  to  smooth  demand,  ability  to  handle  supply  chain, 
number  of  suppliers. 

•  Supply  Chain  Integration:  Number  of  suppliers,  supplier  release  system(from  inventory  levels  or 
customer  order),  supplier  certification,  sourcing  policies  -short-term  or  long-term,  supplier  quality 
ratings,  delivery,  and  productivity  objectives  and  history,  new  product  development 
responsibility,  responsibility  for  kitting  parts,  C-stock  replenishment  efficiency,  supplier  material 
scrap  and  rework,  supplier  cost-saving  ideas  implemented,  supplier  knowledge  of  lean. 

•  Quality  System  Deployment:  Quality  certification,  quality  process  and  measurement  at  each 
process  and  for  each  product,  scrap  and  rework,  problem  solving  process,  product  and  customer 
quality  data,  quality  ratings,  new  product  startup  process,  continuous  improvement  environment, 
degree  of  focus  on  customer  satisfaction  ,  implementation  of  best  practices,  degree  operational 
strategies  are  linked  to  coiporate  strategy,  total  quality  system  well-developed  and  deployed. 

The  RPA  has  an  Assessment  Questionnaire  that  asks  an  additional  set  of  20  questions  whose  answer  is 
yes  or  no.  More  yeses  equate  to  the  more  lean  a  company  is.  The  RPA  has  been  used  to  assess  over  150 
operations  on  400  different  plant  tours  including  Donnelly  Electronics,  Eaton  Corporation's  Aeroquip 
Group,  Haworth's  office  furniture  plants,  a  Lockheed  Martin  division  and  Seagate  Technology.  The  RPA 
can  be  used  as  a  quick  guide  to  assess  competitors  or  suppliers  business  as  well  as  an  in-house  evaluation. 
The  average  score  is  55  out  of  a  possible  121  for  the  assessment  questions.  The  average  number  of  yeses 
for  the  lean  questionnaire  is  seven.  The  RPA  is  conducted  in  the  following  manner.  Prior  to  the  tour  or 
visit,  research  is  done  on  the  company’s  financial  status,  website  and  other  information  using  publicly 
available  resources.  Additionally  one  become  familiar  with  the  type  of  industry  they  will  tour.  A  RPA 
team  should  be  made  up  of  a  variety  of  experience  levels  and  expertise.  The  team  should  not  take  notes 
on  the  tour  but  each  team  member  should  be  responsible  for  a  few  categories  to  make  some  special 
mental  notes.  Directly  after  the  tour  the  team  meets  and  fills  out  individual  questionnaires  and  then 
discusses  high  points  and  low  points  of  the  company. 

The  Rapid  Industrial  Plant  Assessment  (RIPA)  developed  for  this  project  utilized  the  RPA  as  a  guide  to 
the  types  of  questions  and  structure  of  the  assessment.  The  RPA  has  been  a  proven  industrial  tool  and  the 
RIPA  has  used  that  experience  and  knowledge  and  tailored  it  to  meet  the  needs  of  the  AFRL.  The  RIPA 
has  27  targeted  items  that  address  nine  categories  that  collapse  into  three  measures  of  industrial 
preparedness.  The  nine  categories  are  Market  Analysis,  Supply  Chain,  Customer  Satisfaction,  Profit, 
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Cash  Flow,  Capital,  Leadership,  Quality,  and  Lean  Operations.  The  first  three  listed  previously  combine 
to  rate  a  company's  Value  Chain.  The  next  three  rate  a  company's  Financial  performance  and  the  last 
three  rate  a  company's  Operation  Excellence.  The  27  questions  have  three  possible  answers  0  (No),  1 
(Somewhat),  2(Yes).  The  nine  categories  are  rated  on  a  1  to  5  Likert  Scale  where  1  is  Bad  and  5  is  World 
Class.  The  category  ratings  are  mathematically  related  to  the  answers  to  the  27  questions  and  displayed 
in  a  three  dimensional  bubble  graph.  Another  difference  is  that  in  order  to  accurately  rate  a  plant,  the 
RIPA  team  must  reach  a  consensus  rating  of  the  plant  in  each  of  the  nine  categories  above  after  each 
individual  has  completed  a  RIPA  questionnaire. 

Reference 

Goodsen,  Eugene  R.,(2002)  "Read  a  Plant-Fast",  Harvard  Business  Review,  105-113. 

Goodsen,  Eugene  R.,  "Rapid  Plant  Assessment",  http://webuser.bus.umich.edu/Organizations/rpa 


Baldridge  National  Quality  Program 

The  Malcom  Baldridge  National  Quality  Award  was  signed  into  law  on  August  20,  1987.  The  Award  is 
named  for  Malcolm  Baldridge,  who  served  as  Secretary  of  Commerce  from  1981  until  his  death  in  a 
rodeo  accident  in  1987.  Its  intent  is  to  make  a  national  quality  award  program  in  the  United  States  to  help 
improve  quality  versus  our  foreign  competitors.  The  findings  and  purposes  of  the  law  state  that  process 
and  product  quality  have  been  challenged  strongly  by  foreign  competition,  strategic  planning  for  quality 
improvement  programs  are  essential  to  the  wellbeing  of  the  Nation's  economy.  Improved  management 
understanding  of  the  factory  floor  and  greater  emphasis  on  statistical  process  control  can  lead  to  dramatic 
improvements  in  the  cost  and  quality  of  products. 

The  concept  of  quality  improvement  is  directly  applicable  to  small  companies  as  well  as  large  and  to 
service  industries  and  the  public  sector.  In  order  to  be  successful  quality  programs  must  be  management- 
led  and  customer  oriented.  Several  major  industrial  nations  have  successfully  copied  rigorous  private- 
sector  quality  audits  with  national  awards  giving  special  recognition  to  those  enterprises  the  audits 
identify  as  the  very  best.  A  National  quality  award  of  this  kind  the  United  States  helps  to  improve  quality 
and  productivity.  It  stimulates  American  companies  to  improve  quality  by  recognizing  achievements  of 
those  companies  that  improve  the  quality  of  their  goods  and  services,  establishing  guidelines  and  criteria 
that  can  be  used  by  business,  industry  and  government.  It  also  provides  specific  guidance  for  other 
American  organizations  that  wish  to  learn  how  to  manage  for  high  quality.  The  National  Institute  of 
Standards  and  Technology  (NIST)  heads  this  program  to  promote  and  recognize  operational  performance 
excellence.  NIST  is  non-regulatory  agency  within  the  US  Department  of  Commerce.  Categories  and 
questions  from  the  Baldridge  award  were  reviewed  and  utilized  in  initial  drafts  of  the  RIPA.  This  was 
done  to  anchor  the  RIPA  instrument  qualitatively  to  a  known  instrument  as  well  as  to  provide  input  to  an 
initial  list  of  over  200  questions  considered  for  the  RIPA. 

Reference 

"Baldridge  National  Quality  Program".  Baldridge  National  Quality  Program.  NIST.  July  2  2007. 
http://baldridge.nist.gov/  June  30  2007. 

"National  Institute  of  Standards  and  Technology".  National  Institute  of  Standards  and 
Technology.  June  29,  2007.  http://www.nist.gov/  July  2  2007. 
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SYMLOG 


SYMLOG  is  an  acronym  for  the  Systematic  Multiple  Level  Observation  of  Groups.  It  is  a  "system  for  the 
study  of  groups  in  the  sense  that  it  consists  of  a  number  of  different  parts,  integrated  to  serve  the  purpose 
of  making  a  particular  group  easier  to  understand  and  work  with."  (Bales,  Cohen,  Williamson,  1979)  It  is 
a  set  of  methods  for  the  study  of  groups  of  any  kind,  but  basically  small  natural  groups  such  as  families, 
teams  or  classroom  groups.  The  SYMLOG  adjective  rating  instrument  consists  of  26  ratings  which  are 
traditionally  scored  on  a  scale  of  0,  1,  2.  The  instrument  display's  its  results  (from  either  scoring  system) 
graphically  in  a  three  dimensional  bubble  plot.  This  type  of  plot  is  used  in  the  graphical  display  of  the 
RIPA  instrument  where  the  size  of  the  bubble  corresponds  to  the  Financial  stability  of  the  company  of 
interest  and  the  x  and  y  axes  correspond  to  Operational  Excellence  and  Value  Chain.  The  RIPA  also 
borrows  the  0,  1,  2  scale  from  this  instrument.  The  27  RIPA  items  are  rated  using  0,  1,  2  where  0 
corresponds  to  No,  1  corresponds  to  Somewhat  and  2  is  Yes.  The  original  SYMLOG  instrument  used  a 
two-point  scale  of  only  yes  or  no,  similar  to  the  RPA.  Research  through  the  years  led  the  purveyors  of  the 
SYMLOG  instrument  to  change  a  the  three  point  scale.  This  was  the  basis  for  the  three-point  scale  for 
the  RIPA. 

Reference 

Bales,  R.  F.,  Cohen,  Stephen  P.,  Williamson,  S.  A.  SYMLOG.  1979.  The  Free  Press. 


University  of  Tennessee  Development  Staff 

Development  of  the  RIPA  and  training  was  led  by  Allen  Pannell  with  extensive  participation  by  Dr. 
Mandyam  Srinivasan,  Dr.  Dan  Flint  and  Dr.  Elaine  Seat.  Their  years  of  practical  and  academic 
experience  contributed  to  the  initial  set  of  questions  and  categories  and  ultimately  to  the  selection  of  the 
final  questions  and  categories.  Sets  of  questions  used  in  Mr.  Pannell’ s  consulting  practice  also  were  used. 
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APPENDIX  D:  MRL  AND  RIPA  QUESTIONS  COMPARISON 

The  following  is  a  brief  summary  of  the  MRL  questions  compared  to  the  27  RIPA  questions.  In  a  one 
sentence  summary  comparing  the  two  methods  of  evaluation,  the  RIPA  takes  a  broad  stroke  look  at  the 
company/contractor  as  a  whole  where  as  the  MRL  questions  are  small  benchmarks  specifically  regarding 
the  specific  system  or  product.  The  following  bullets  give  some  examples  for  comparison. 

•  The  MRL  questions  never  assess  the  company’s  financials  as  a  whole,  only  their  ability  to  fund 
the  current  program  (cost  and  funding  thread).  There  are  some  questions  about  cost  improvement 
but  all  regarding  cost  reduction  within  this  certain  program. 

•  The  RIPA  is  a  snapshot  in  time  where  as  the  MRL  questions  change  as  the  level  changes.  For 
instance  in  MRL  4  in  the  Process  Capability  and  Control  thread  the  question  is  asked  if  “yield  and 
rates  assessment  on  proposed  processes  complete”  and  in  MRL  8  the  question  asks  if  “yield  and 
rates  required  to  begin  LRIP  verified  using  SDD  articles.” 

•  The  MRLs  focus  on  training  programs  and  whether  or  not  personnel  are  properly  trained  whereas 
the  RIPA  asks  about  the  personnel’s  ability  to  affect  the  quality  of  a  product.  The  RIPA 
addresses  whether  or  not  personnel  in  the  plant  are  focused  on  quality  and  if  those  personnel  are 
empowered  to  do  anything  about  poor  quality.  In  the  Personnel  thread,  the  questions  are 
concentrated  on  training  benchmarks  to  reach  the  appropriate  MRA  level. 

•  The  RIPA  addresses  market  for  the  system  or  product.  The  MRL  questions  do  not  address  the 
possible  market  or  marketing  ability  of  the  contractor/company. 

•  The  MRLs  have  some  questions  dealing  with  the  company’s  suppliers  and  how  they  are  managed 
and  does  not  address  customer  relationships.  The  RIPA  briefly  mentions  suppliers  in  question  14 
but  questions  the  contractor/company’s  customer  relationships  in  three  separate  questions. 

•  The  MRL  discusses  process  flow,  but  only  in  the  context  of  that  particular  line  and  those 
questions  change  with  each  MRL.  The  RIPA  asks  if  the  workplace  in  general  is  conducive  to 
good  process  flow. 

•  The  MRL  never  directly  addresses  the  value  chain. 
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APPENDIX  E:  COMPANY  A  VISIT  SUMMARY 


Introduction 

Company  A  is  a  privately  owned,  small  business  that  provides  specialty  materials  as  a  supplier  of 
aerospace/space/and  military  application  parts.  They  provide  high  strength,  custom-designed  lightweight 
components.  They  have  a  history  of  designing  and  producing  low  volume  components  that  can  be  hand 
constructed.  However,  they  are  presently  pursuing  certification  of  aerospace  parts  that  can  become  a 
standard  product  for  the  aerospace  MRO  industry. 

Company  A  is  the  result  of  a  large  company  spin-off  of  this  technology.  Many  of  the  professional  staff 
have  relocated  to  Company  A  to  continue  working  with  the  technology.  Although  Company  A  is  new  to 
this  business,  the  staff  has  a  long  history  and  expertise  in  the  R&D  application  of  the  technology.  Five 
years  ago,  Company  A  was  a  manufacturer  of  crude  products  for  military  field  application.  They 
continue  this  line  of  business.  Their  specialty  materials  business  is  new  and  derived  from  the  technology 
spin-off.  There  is  no  connection  between  the  original  company  mission  and  today’s  mission. 

They  are  presently  ramping  up  from  a  research  house  to  a  product  supplier  of  their  proprietary 
technology.  Company  A  is  moving  from  a  pure  research  facility  to  research  and  production.  They  are 
presently  growing  their  manufacturing  capability.  They  plan  to  increase  their  gross  by  500%  in  five 
years. 

Company  A  is  ISO  9001  and  AS9004  certified. 

Interview  and  Tour  Summary 

Company  A’s  management  team  has  developed  an  extensive  business  plan.  The  purpose  of  this  plan  is  to 
secure  necessary  funding  capital  for  improvements  required  for  production.  Company  A’s  strength  is 
technical  problem  solving.  They  have  little  expertise  with  production. 

Management  Debrief 

The  management  team  was  interviewed  regarding  1)  how  they  have  changed  in  the  last  five  years  and  2) 
what  they  believe  to  be  their  biggest  challenges  and  obstacles. 

Methodology 

The  Company  A  management  team  evaluated  their  company  in  four  different  ways.  Each  data  set  was 
provided  independently  by  each  member  of  the  Company  team  in  paper  and  pencil  mode.  The  four  data 
sets  are: 


Data  Index  6:  Scoring  of  the  27  items  for  present  state. 

Data  Index  7:  Scoring  of  the  company  on  the  nine  categories  in  its  present  state  (1-5  scale). 

Data  Index  8:  Scoring  of  the  company  on  the  27  items  five  years  ago. 

Data  Index  9:  Scoring  of  the  company  on  the  nine  categories  five  years  ago  (1-5  scale). 

The  management  team  did  not  discuss  their  ratings  or  reach  consensus. 

The  data  were  input  into  the  RIPA  assessment  spreadsheet  by  the  UT  team  of  researchers. 

The  UT  team  arrived  at  their  RIPA  results  by  each  team  member  independently  answering  the  27  items 
(data  index  3)  and  then  reaching  consensus  as  a  team  on  the  final  score  for  each  of  the  nine  categories 
(data  index  5). 
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Data 

RIPA  Data 


Figure  13  provides  the  nine  item  scores  as  generated  by  the  RIPA  calculator  for  the  Company  A 
management  team’s  input  of  the  27  items.  Figure  14  provides  the  UT  team’s  scores  for  the  nine  items 
generated  by  the  RIPA  calculator  and  their  consensus  scores. 


Figure  13.  Company  A  -  Management’s  Rating  From  Input  of  27  Items  (Data  Index  10) 
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Market  Analysis 
Supply  Chain 

Customer  Satisfaction 

Leadership 

Quality 


Figure  14.  Company  A  -  UT  Team’s  Evaluation  From  Consensus  (Data  Index  5) 


Figure  15  provides  a  graphical  comparison  using  the  RIPA  bubble  chart  representation  of  the 
management  team’s  evaluation  of  now  and  five  years  ago,  and  the  evaluation  from  the  UT  team. 
Company  A  managers  evaluated  their  company  using  the  27  items  both  now  and  five  years  prior. 

Company  A  was  in  a  unique  position  when  evaluating  themselves  five  years  prior  because  they  may  have 
the  same  name,  but  are  not  the  same  company  and  they  now  have  a  different  mission.  However,  their 
ability  to  change  the  culture  to  a  new  mission  and  type  of  work  will  be  important  to  their  success. 

The  data  used  in  the  Company  A  management  charts  comes  directly  from  their  scores  on  the  27  items. 

Qualitative  Data 

The  management  team  responded  to  two  questions  to  provide  qualitative  data.  The  intent  of  this 
questioning  was  to  gather  supporting  information  for  1)  the  five  year  previous  RIPA  input  and  2)  the 
present  RIPA  from  a  casual  dialogue  with  the  Company  A  management  team. 

What  has  changed  over  the  last  five  years? 

•  Their  facility  had  a  completely  different  mission  and  product  that  only  required  crude  techniques 
for  manufacturing  and  their  new  mission  requires  precision  techniques. 

What  challenges/obstacles  face  your  company? 

•  Stay  creative  while  also  doing  production. 

•  Create  product  in  a  reliable  way. 
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Operational  Excellence 

Figure  15.  Company  A  -  All  RIPA  Evaluations 
(Company  A  Now,  Company  A  Five  Years  Ago  and  UT  Team) 

Analysis 

Company  A  is  moving  from  a  research  facility  to  a  manufacturing  facility  with  specific  components  they 
manufacture.  They  have  a  history  of  producing  one  of  a  kind  single  components  to  specification.  They 
have  been  actively  developing  a  marketing  plan  in  an  attempt  to  become  a  manufacturing  facility. 

Summary  of  Results 

Figure  15  facilitates  a  comparison  of  all  scores  from  the  RIPA  survey  of  Company  A.  This  graphic 
suggests: 

•  The  UT  team  estimated  the  financial  viability  of  Company  A  higher  than  the  management  team. 
Company  A  is  privately  owned,  and  the  difficulty  in  making  capital  expenditures  necessary  for 
the  business  expansion  was  perceived  to  be  greater  by  those  in  the  company  than  the  UT  team. 
Although  the  UT  team  financial  circle  is  larger  than  the  management  team’s  circle,  both  are 
smaller  than  the  reference  circle  (midpoint). 

•  The  UT  team  had  scores  that  were  most  dissimilar  to  the  Company  A  management  team’s  scores 
on  the  factors  that  make  up  Operational  Excellence.  Company  A  is  undergoing  a  change  and  as 
such,  they  had  inconsistent  scores  that  reflect  their  range  of  performance  on  Leadership,  Lean, 
and  Quality.  Depending  on  the  specific  task,  they  had  a  range  of  performance. 

•  The  management  team  and  the  UT  team  both  scored  Company  A  the  same  for  the  Value  Stream 
construct.  Company  A  has  produced  specialty  products  that,  although  perhaps  late  on  delivery 
and  expensive,  met  the  customer’s  performance  expectations.  Company  A  has  received  positive 
feedback  from  customers  and  has  a  marketing  plan  for  future  improved  financial  performance. 
Thus,  they  have  scores  at  or  slightly  above  the  reference  line  in  the  Value  Stream  component. 
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APPENDIX  F:  COMPANY  C  VISIT  SUMMARY 


Introduction 

Company  C  makes  industrial  adhesives  for  the  non-wovens  market.  They  use  a  limited  number  of  raw 
ingredients  to  produce  many  different  adhesive  products.  These  products  are  provided  to  the  customer  in 
several  different  forms.  Company  C  produces  commodity  adhesive  products  and  their  key  process  index 
is  cost  per  pound  where  most  of  that  cost  is  associated  with  labor.  Company  C  is  a  stand-alone 
manufacturing  facility  with  no  marketing,  sales,  financial  or  human  resources  on  site.  Those  functions 
are  performed  on  a  coiporate  wide  basis  from  a  central  organization.  Company  C  is  a  cost  center  for  their 
parent  company. 

Company  C  is  required  to  make  a  6%  improvement  in  productivity  each  year  where  the  metric  is  pounds 
per  labor  hour. 

Interview  and  Tour  Summary 

Company  C  has  a  continuous  flow  into  it’s  facility  of  raw  materials  to  make  adhesives  and  a  continuous 
flow  out  of  finished  adhesive  product.  The  atmosphere  is  hot  and  unpleasant  to  many  potential 
employees.  The  work  is  not  difficult  and  pays  low  wages.  However,  employees  with  out  educational 
credentials  have  the  opportunity  for  steady  work. 

We  were  debriefed  by  the  plant  manager.  This  facility  is  undergoing  improvements  to  its  process.  The 
employees  are  learning  data-driven  process  control.  The  percent  of  quality  problems  has  been  reduced 
from  10  percent  to  two  percent  within  the  last  five  years  due  to  process  monitoring  and  improvements. 

Company  C  is  focused  on  their  customers  and  understands  their  place  in  the  supply  chain.  They  believe 
that  this  customer  focus  may  be  carried  to  a  fault  as  it  affects  batch  size,  product  form,  and  delivery  in  a 
manner  that  is  not  cost  effective.  Understanding  customer  requirements  and  product  mix  is  a  focus  area 
for  improvement. 

Management  Debrief 

The  plant  manager  was  interviewed  regarding  1)  how  they  have  changed  in  the  last  five  years  and  2)  what 
they  believe  to  be  their  biggest  challenges  and  obstacles. 

Methodology 

The  Company  C  management  team  evaluated  their  company  in  four  different  ways.  Each  data  set  was 
provided  independently  by  each  member  of  the  Company  team  in  paper  and  pencil  mode.  The  four  data 
sets  are: 

•  Data  Index  6: 

•  Data  Index  7 : 
scale). 

•  Data  Index  8: 

•  Data  Index  9: 
scale). 

The  management  team  did  not  discuss  their  ratings  or  reach  consensus. 

The  data  was  input  into  the  RIPA  spreadsheet  by  the  UT  team  of  researchers. 

The  UT  team  arrived  at  their  RIPA  results  by  each  team  member  independently  answering  the  27  items 
(data  index  3)  and  then  reaching  consensus  as  a  team  on  the  final  score  for  each  of  the  nine  categories 
(data  index  5). 


Scoring  of  the  27  items  for  present  state. 

Scoring  of  the  company  on  the  nine  categories  in  its  present  state  (one  through  five 

Scoring  of  the  company  on  the  27  items  five  years  ago. 

Scoring  of  the  company  on  the  nine  categories  five  years  ago  (one  through  five 
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Allen  Pannell  knew  this  company  from  prior  consulting  work.  For  this  visit,  Maria  Weese  and  Elaine 
Seat  served  as  the  evaluators.  The  UT  team  RIPA  score  was  generated  from  their  input  and  consensus 
scores.  Allen  Pannell  served  as  the  expert  evaluator  and  his  input  is  provided  in  Figure  18  for  comparison 
with  the  management  team  and  UT  team  scores. 

Data 

RIPA  Data 


Figure  16  provides  the  nine  item  scores  as  generated  by  the  RIPA  calculator  for  the  Company  C 
management  team’s  input  of  the  27  items.  Figure  17  provides  the  UT  team’s  for  the  nine  items  as 
generated  by  the  RIPA  calculator  of  the  27  items  and  the  consensus  scores. 


Figure  16.  Company  C  -  Management’s  Rating  From  Input  of  27  Items  (Data  Index  10) 
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Figure  17.  Company  C  -  UT  Team’s  Evaluation  From  Consensus  (Data  Index  5) 


Figure  18  provides  a  graphical  comparison  of  the  management  team’s  evaluation  of  now  and  5  years  ago, 
and  the  evaluation  from  the  UT  team. 

The  data  used  in  the  Company  C  management  charts  comes  directly  from  their  scores  on  the  27  items. 

Qualitative  Data 

The  management  team  responded  to  two  questions  to  provide  qualitative  data.  The  intent  of  this 
questioning  was  to  gather  supporting  information  for  1)  the  five  year  previous  RIPA  input  and  2)  the 
present  RIPA  from  a  casual  dialogue  with  the  Company  C  management  team. 

What  has  changed  over  the  last  five  years? 

•  Now  there  is  a  vision  and  tools  to  execute  change. 

•  Making  data  driven  decisions  (measures,  determine  where  you  are  and  where  you  want  to  go). 
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Company  C 


Figure  18.  Company  C  -  All  RIPA  Evaluations  (Company  C  and  UT  Team) 


What  challenges/obstacles  face  your  company  ? 

•  Understanding  change  doesn’t  happen  overnight. 

•  Having  team  leaders  who  can  understand  and  execute  continual  change. 

•  Having  validated  procedures. 

Summary  of  Results 

The  RIPA  data  suggests  the  following  conclusions: 

•  The  bubble  in  Figure  1 8  by  the  management  team  is  significantly  higher  in  Operational 
Excellence  and  the  Value  Stream  than  the  UT  team  and  the  expert  rating.  This  quantitative  data 
suggests  that  the  company  faced  a  challenge  of  having  all  players  understand  change  and  exactly 
where  the  company  was.  This  was  demonstrated  to  the  UT  team  in  individual  interviews  when 
the  lower  level  managers  did  not  have  the  same  opinion  and  attitudes  toward  the  company  as  the 
top  management.  This  inflated  management  score  is  representative  of  the  disconnect  between 
reality  of  the  company  and  what  was  observed. 

•  The  UT  team  and  the  expert  reached  the  same  RIPA  scores.  This  indicates  that  the  RIPA  is 
providing  the  same  feedback  from  trained  observers  and  experts  with  inside  knowledge  of  a 
company.  (See  Company  D  for  this  same  circumstance  and  result). 
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•  Company  C  has  been  on  a  course  of  Lean  training  and  business  improvement  for  last  several 
years.  This  is  manifest  in  the  improved  scored  from  five  years  ago  and  the  present. 

•  Management  receives  funding  for  capital  projects  from  their  central  office  in  competition  with 
other  cost  centers.  As  Lean  techniques  and  data  driven  decision  making  have  been  implemented, 
the  Company’s  ability  to  receive  funding  for  capital  improvements  has  improved,  along  with 
production  efficiencies. 
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APPENDIX  G:  COMPANY  D  VISIT  SUMMARY 


Introduction 

Company  D  makes  industrial  adhesives  for  the  non-wovens  market.  They  use  a  limited  number  of  raw 
ingredients  to  produce  over  100  different  adhesive  products.  They  produce  no  retail  products.  Company 
D  produces  commodity  adhesive  products  and  their  key  process  index  is  cost  per  pound  where  most  of 
that  cost  is  associated  with  labor.  Company  D  is  a  stand-alone  manufacturing  facility  with  no  marketing, 
sales,  financial  or  human  resources  on  site.  Those  functions  are  performed  on  a  corporate  wide  basis 
from  a  central  organization.  Company  D  is  a  cost  center  for  their  parent  company. 

Company  D  is  required  to  make  a  six  percent  improvement  in  productivity  each  year  where  the  metric  is 
pounds  per  labor  hour. 

Interview  and  Tour  Summary 

Company  D’s  employees  were  very  receptive  and  interviews  were  relaxed  and  informative.  Their  shop 
appeared  orderly,  organized,  well  lit  and  well  marked.  There  were  metrics  posted  not  only  in  the  meeting 
room  but  on  the  shop  floor  and  in  the  break  room.  Additionally,  there  was  a  kiosk  that  rotated  through 
production  information,  quality  information,  key  process  indicators  and  other  information  in  the  main 
lobby  and  in  the  break  room.  Company  D  did  a  very  good  job  of  communicating  within  their  facility  to 
their  employees.  Their  facility  is  laid  out  to  promote  efficiency,  with  all  of  the  receiving  in  one  spot  and 
the  shipping  in  for  all  the  different  product  lines  in  another. 

Company  D  is  dedicated  to  quality  and  leanness,  although  they  admit  sometimes  they  feel  too  lean.  They 
employ  a  seven  step  procedure  for  problem  solving  throughout  the  entire  organization.  Company  D  has  a 
unique  management  system  in  that  there  are  no  shift  supervisors  only  team  leaders.  These  team  leaders 
work  on  the  day  shift  and  do  not  directly  supervise  the  first  shift.  Each  team  lead  has  other 
responsibilities  beyond  just  their  team  at  an  organizational  level.  The  shifts  are  responsible  entirely  for 
themselves. 

Management  Debrief 

The  management  team  was  interviewed  regarding  1)  how  they  have  changed  in  the  last  five  years  and  2) 
what  they  believe  to  be  their  biggest  challenges  and  obstacles. 

Methodology 

The  Company  D  management  team  evaluated  their  company  in  four  different  ways.  Each  data  set  was 
provided  independently  by  each  member  of  the  Company  team  in  paper  and  pencil  mode.  The  four  data 
sets  are: 

•  Data  Index  6:  Scoring  of  the  27  items  for  present  state. 

•  Data  Index  7:  Scoring  of  the  company  on  the  nine  categories  in  its  present  state  (one  through  five 
scale). 

•  Data  Index  8:  Scoring  of  the  company  on  the  27  items  five  years  ago. 

•  Data  Index  9:  Scoring  of  the  company  on  the  nine  categories  five  years  ago  (1  through  5  scale). 
The  management  team  did  not  discuss  their  ratings  or  reach  consensus. 

The  data  was  input  into  the  RIPA  assessment  spreadsheet  by  the  UT  team  of  researchers. 

The  UT  team  arrived  at  their  RIPA  results  by  each  team  member  independently  answering  the  27  items 
(data  index  3)  and  then  reaching  consensus  as  a  team  on  the  final  score  for  each  of  the  nine  categories 
(data  index  5). 
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Allen  Pannell  knew  this  company  from  prior  consulting  work.  For  this  visit,  Maria  Weese  and  Elaine 
Seat  served  as  the  evaluators.  The  UT  team  RIPA  score  was  generated  from  their  input  and  consensus 
scores.  Allen  Pannell  served  as  the  expert  evaluator  and  his  input  is  provided  in  Figure  21  and  Figure  18 
for  comparison  with  the  management  team  and  UT  team  scores. 

Data 

RIPA  Data 


Figure  19  provides  the  nine  item  scores  as  generated  by  the  RIPA  calculator  for  the  Company  D 
management  team’s  input  of  the  27  items.  Figure  20  provides  UT  evaluation  team’s  scores  for  the  nine 
items  as  generated  by  the  RIPA  calculator  of  the  27  items  and  the  consensus  scores. 


Market  Analysis 
Supply  Chain 
Customer  Satisfaction 
Leadership 
Quality 
Lean  Operations 
Profit 
Cash  Flow 
Capital 


0 


0 


1  2  3  4  5 


Figure  19.  Company  D  -  Management’s  Nine  Item  Rating  From  Input  of  27  Items  (Data  Index  10) 
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Figure  20.  Company  D  -  Evaluation  -  UT  Team’s  Evaluation  From  Consensus  (Data  Index  5) 


Figure  18  provides  a  graphical  comparison  of  the  management  team’s  evaluation  of  now  and  five  years 
ago,  and  the  evaluation  from  the  UT  team. 

Qualitative  Data 

The  management  team  responded  to  two  questions  to  provide  qualitative  data.  The  intent  of  this 
questioning  was  to  gather  supporting  information  for  1)  the  five  year  previous  RIPA  input  and  2)  the 
present  RIPA  from  a  casual  dialogue  with  the  Company  D  management  team. 

What  has  changed  over  the  last  five  years? 

•  Development  of  the  staff. 

•  Lots  of  benchmarking  to  develop  optimum  processes. 
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Company  D 


Figure  21.  Company  D  -  All  RIPA  Evaluations 
(Company  D  Now,  Company  D  Five  Years  Ago  and  UT  Team) 


What  challenges/obstacles  face  your  company? 

•  Getting  everyone  on  the  same  page. 

•  Adequate  staffing. 

•  Getting  our  suppliers  evaluated  and  part  of  our  process. 

•  Being  proactive  and  not  reactive. 

Analysis 

Company  D  received  a  high  rating  in  every  category  with  their  highest  of  4.5  being  in  capital.  Since 
Company  D  can  obtain  capital  from  their  coiporate  office  and  they  operate  with  such  efficiency  their 
operational  excellence  is  high. 

Company  D  has  taken  a  structured  approach  to  improving  their  operational  processes  for  several  years. 
They  understand  that  their  product  is  a  commodity  and  they  also  understand  their  place  in  the  supply 
chain  for  their  downstream  customers.  As  such,  they  have  focused  on  eliminating  any  waste.  This  focus 
is  evident  from  observation,  impromptu  talks  with  shop  personnel,  and  posted  data. 

The  financial  data  presented  by  Company  D  management  supports  their  improvement  in  processes  and 
eliminating  waste  as  their  price  per  pound  of  product  has  declined  over  the  last  five  years. 
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Summary  of  Results 

Figure  21  facilitates  a  comparison  of  all  scores  from  the  RIPA  survey  of  Company  D.  This  graphic 
suggests: 

•  Company  D  shows  in  its  RIPA  evaluations  big  improvements  over  the  past  five  years. 

•  The  UT  team  and  the  expert  reached  the  same  RIPA  scores.  This  indicates  that  the  RIPA  is 
providing  the  same  feedback  from  trained  observers  and  experts  with  inside  knowledge  of  a 
company.  (See  Company  C  for  this  same  circumstance  and  result). 

•  Company  D  has  been  on  a  course  of  Lean  training  and  business  improvement  for  last  several 
years.  This  is  manifest  in  the  improved  scored  from  five  years  ago  and  the  present. 

•  Management  receives  funding  for  capital  projects  from  their  central  office  in  competition  with 
other  cost  centers.  As  Lean  techniques  and  data  driven  decision  making  have  been  implemented, 
the  Company’s  ability  to  receive  funding  for  capital  improvements  has  improved,  along  with 
production  efficiencies. 
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APPENDIX  H:  COMPANY  E  VISIT  SUMMARY 


Introduction 

Company  E  is  a  Tier  III,  low  volume  high  mix  military/aerospace  and  medical/industrial  contract 
manufacturing  facility.  They  primarily  operate  in  electronics  and  communications.  Much  of  their  work  is 
in  the  defense  industry.  Company  E  has  over  150  different  end  products  per  month  and  has  over  100,000 
different  products  in  their  product  catalogue.  Company  E’s  sales  have  grown  by  almost  40  percent  in  the 
past  two  years. 

Company  E  is  part  of  a  larger  public  coiporation  that  is  active  in  acquiring  other  companies. 

Company  E  has  a  focus  on  operational  excellence  and  waste  reduction.  They  use  price  of 
nonconformance  as  a  key  process  indicator  measured  as  a  percent  of  their  business  each  year.  They  have 
a  goal  of  a  20  percent  year  over  year  waste  reduction.  Company  E  has  their  own  lean  program  headed  by 
a  council  that  meets  quarterly  to  share  best  practices  within  the  company.  On  their  site  alone,  there  are 
eight  Six  Sigma  certified  Black  or  Green  belts.  90  percent  of  the  facility  has  been  through  eight  hours  of 
lean  training  and  the  use  of  Six  Sigma  tools.  Company  E  is  ISO  and  DoD  certified. 

Company  E  focuses  on  customer  service  by  not  letting  a  customer  get  lost  in  the  mix  and  by 
manufacturing  things  that  no  one  else  wants  to  make.  They  look  to  maintain  a  few  large  and  secure 
customers.  Their  marketing  is  based  on  horizontal  expansion,  they  look  to  do  so  well  on  a  first  project 
that  the  customer  always  comes  back. 

Interview  and  Tour  Summary 

Company  E’s  shop  is  organized  like  factories  within  a  factory.  Work  cells  have  cut  transfer  time  on  the 
shop  floor  and  allow  for  visual  management.  Operators  always  work  in  the  same  cells  and  are  trained  to 
do  the  jobs  in  those  cells.  Overall  the  shop  was  neat,  there  were  metrics  posted  and  operators  very  nice  to 
greet  us  as  we  toured.  There  were  some  secure  areas  where  product  can  be  manufactured  separately  to 
meet  proprietary  and/or  security  requirements. 

Employees  were  interviewed.  The  company  is  like  a  family  and  is  one  of  the  few  high  tech  stable 
employers  in  the  area.  The  employees  are  satisfied  and  enjoy  the  family  atmosphere. 

Management  Debrief 

The  plant  manager  was  interviewed  regarding  1)  how  they  have  changed  in  the  last  five  years,  2)  what 
they  believe  to  be  their  biggest  challenges  and  obstacles,  and  3)  what  they  believe  their  strengths  to  be. 

Methodology 

The  Company  E  management  team  evaluated  their  company  in  four  different  ways.  Each  data  set  was 
provided  independently  by  each  member  of  the  Company  team  in  paper  and  pencil  mode.  The  four  data 
sets  are: 


•  Data  Index  6:  Scoring  of  the  27  items  for  present  state. 

•  Data  Index  7:  Scoring  of  the  company  on  the  nine  categories  in  its  present  state  (1  through  5 
scale). 

•  Data  Index  8:  Scoring  of  the  company  on  the  27  items  five  years  ago. 

•  Data  Index  9:  Scoring  of  the  company  on  the  nine  categories  five  years  ago  (1  through  5  scale). 
The  management  team  did  not  discuss  their  ratings  or  reach  consensus. 

The  data  was  input  into  the  RIPA  assessment  spreadsheet  by  the  UT  team  of  researchers. 
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The  UT  team  arrived  at  their  RIPA  results  by  each  team  member  independently  answering  the  27  items 
(data  index  3)  and  then  reaching  consensus  as  a  team  on  the  final  score  for  each  of  the  nine  categories 
(data  index  5). 

Data 


RIPA  Data 

Figure  22  provides  the  nine  item  scores  as  generated  by  the  RIPA  calculator  for  the  Company  E 
management  team’s  input  of  the  27  items.  Figure  23  provides  the  UT  team’s  scores  for  the  nine  items  as 
generated  by  the  RIPA  calculator  of  the  27  items  and  the  consensus  scores. 

Company  E 


Market  Analysis 
Supply  Chain 
Customer  Satisfaction 
Leadership 
Quality 
Lean  Operations 
Profit 
Cash  Flow 
Capital 


1 4.2 


1 3.6 


1 3.4 


1 4.2 


: 

4.8 


Figure  22.  Company  E  -  Management’s  Rating  From  Input  of  27  Items  (Data  Index  10) 


47 


Figure  23.  Company  E  -  UT  Team’s  Evaluation  From  Consensus  (Data  Index  5) 


Figure  24  provides  a  graphical  comparison  of  the  management  team’s  evaluation  of  now  and  five  years 
ago,  and  the  evaluation  from  the  UT  team. 

Company  E  was  asked  to  fill  out  a  nine  question  survey  evaluating  itself  in  the  nine  categories  now  and 
five  years  prior.  Additionally  they  were  asked  to  evaluate  themselves  using  the  27  RIPA  questions  for 
now  and  five  years  prior. 

Qualitative  Data 

The  management  team  responded  to  two  questions  to  provide  qualitative  data.  The  intent  of  this 
questioning  was  to  gather  supporting  information  for  1)  the  five  year  previous  RIPA  input  and  2)  the 
present  RIPA  from  a  casual  dialogue  with  the  Company  E  management  team. 

What  has  changed  over  the  last  five  years? 

•  New  management  structure. 

•  Total  turnover  in  management  staff. 

•  Automation  at  all  levels. 


48 


Company  E 


c 

'ro 

sz 

O 

<1) 

3 

(0 

> 


Operational  Excellence 

Figure  24.  Company  E  -  All  RIPA  Evaluations 
(Company  E  Now,  Company  E  Five  Years  Ago  and  UT  Team) 

What  challenges/obstacles  face  your  company? 

•  Their  supply  chain  of  suppliers. 

•  Empowering  the  workforce  so  management  can  be  efficient. 

•  Making  financial  management  more  of  a  priority. 

What  are  your  strengths? 

•  Customer  Satisfaction. 

•  Don’t  use  anecdotal  evidence  to  make  decisions. 
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Summary  of  Results 

Figure  24  facilitates  a  comparison  of  all  scores  from  the  RIPA  survey  of  Company  E.  This  graphic 
suggests: 

•  The  RIPA  scores  from  the  UT  team  and  the  management  team  are  in  alignment. 

•  Company  E  has  access  to  capital  as  evidenced  by  recent  significant  capital  improvements  in  their 
facility.  These  recent  improvements  are  evidenced  in  the  improved  management  scores  from  five 
years  ago  and  the  present,  and  the  UT  team’s  ability  to  observe  these  improvements. 

•  Company  E  management  reported  that  they  needed  to  better  attend  to  financials,  but  in  their 
RIPA  reporting,  they  provided  high  scores  in  financials. 

•  Company  E  is  an  established  company  that  provided  detailed  information,  access  to  employees 
for  personal  interviews,  and  a  detailed  tour  of  the  facilities.  This  meeting  took  three  hours,  and 
facilitated  the  team  evaluating  Company  E  the  same  as  management’s  evaluation. 

•  Company  E’s  comments  regarding  the  changes  in  their  workplace  from  five  years  ago  are 
reflected  in  their  five  years  ago  and  present  RIPA  evaluations. 
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APPENDIX  I:  COMPANY  F  VISIT  SUMMARY 


Introduction 

Company  F  is  an  employee  owned  conglomerate  consisting  of  four  business  units.  All  of  the  business 
units  manufacture  vibration  measurement  and  balancing  equipment  designed  for  field  use  by  technicians 
in  nuclear  applications,  electrical  applications,  aeronautics  and  materials  testing.  They  expect  10  percent 
profit  and  1 0  percent  growth  annually. 

The  company  has  sales  worldwide  and  has  a  strong  marketing  team  and  product  presence.  They  have 
focus  on  customer  satisfaction  with  their  product  and  accurate  performance  of  their  instrumentation. 

They  have  carefully  analyzed  the  marketplace  and  focused  their  effort  in  niches  where  they  can  be  the 
best. 

Interview  and  Tour  Summary 

The  president  of  the  company  and  marketing  manager  met  with  our  team.  The  president  presented  an 
overview  of  the  company.  It  included  not  only  where  Company  F  is  today,  but  where  they  wish  to  be  in 
the  future.  They  view  themselves  as  problem  solvers  for  their  customers  and  frequently  that  is  how  they 
gain  market  share  in  a  business.  We  toured  their  facility  with  visits  to  engineering,  the  testing  laboratory, 
and  the  manufacturing  floor. 

Management  Debrief 

The  management  team  was  interviewed  regarding  1)  how  they  have  changed  in  the  last  five  years  and  2) 
what  they  would  change  to  see  improved  performance. 

Methodology 

The  Company  F  management  team  evaluated  their  company  in  three  different  ways.  Each  data  set  was 
provided  independently  by  each  member  of  the  Company  team  in  paper  and  pencil  mode.  The  three  data 
sets  are: 

•  Data  Index  6:  Scoring  of  the  27  items  for  present  state. 

•  Data  Index  7:  Scoring  of  the  company  on  the  nine  categories  in  its  present  state  (one  through  five 
scale). 

•  Data  Index  9:  Scoring  of  the  company  on  the  nine  categories  five  years  ago  (one  through  five 
scale). 

The  management  team  did  not  discuss  their  ratings  or  reach  consensus. 

The  data  was  input  into  the  RIPA  spreadsheet  by  the  UT  team  of  researchers. 

The  UT  team  arrived  at  their  RIPA  results  by  each  team  member  independently  answering  the  27  items 
(data  index  3)  and  then  reaching  consensus  as  a  team  on  the  final  score  for  each  of  the  nine  categories 
(data  index  5). 

Data 

RIPA  Data 

Figure  25  provides  the  nine  item  scores  as  generated  by  the  RIPA  calculator  for  the  Company  A 
management  team’s  input  of  the  27  items.  Figure  26  provides  the  UT  team’s  scores  for  the  nine  items  as 
generated  by  the  RIPA  calculator  of  the  27  items  and  the  consensus  scores. 
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Figure  25.  Company  F  -  Management’s  Rating  From  Input  of  27  Items  (Data  Index  10) 


Figure  26.  Company  F  -  UT  Team’s  Evaluation  From  Consensus  (Data  Index  5) 
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Figure  27  provides  a  graphical  comparison  of  the  management  team’s  evaluation  of  now  and  five  years 
ago,  and  the  evaluation  from  the  UT  team. 

The  data  used  in  the  Company  F  management  charts  comes  directly  from  their  scores  on  the  27  items. 

Company  F 


Figure  27.  Company  F  -  All  RIPA  Evaluations 
(Company  F  Now,  Company  F  Five  Years  Ago  and  UT  Team) 

Qualitative  Data 

The  management  team  responded  to  two  questions  to  provide  qualitative  data.  The  intent  of  this 
questioning  was  to  gather  supporting  information  for  1)  the  five  year  previous  RIPA  input  and  2)  the 
present  RIPA  with  verbal  insights  as  to  where  the  company  felt  they  could  improve. 


Did  you  tend  to  rate  yourself  higher  or  lower  now  versus  before? 

•  Definitely  higher  now  than  five  years  ago,  and  way  higher  than  six  years  ago.  We  have  had 
restructuring  that  cut  the  budget  and  increased  productivity  to  meet  same  targets. 

•  Improvements  are  attributed  to  management  approach:  conservative,  better  decision  making; 
they  still  struggle  with  decisions  about  growth. 

•  Have  consequences  now  for  not  meeting  targets. 

•  Management  is  conservative  because  they  remember  time  when  company  was  bleeding 
financially  from  overextending  on  growth. 


53 


•  Management  cut  budgets  and  then  increased  productivity  to  meet  budget.  This  made  a  big 
difference. 


If  there  is  one  thing  you  ’d  change  to  improve  performance  of  your  company,  what  would  it  be? 

•  More  mature  processes  in  manufacturing  and  engineering;  still  too  ad  hoc. 

•  Right  mix  of  new  staff  to  get  to  new  level. 

•  How  to  exploit  new  opportunities. 

•  Formal  LEAN/six  sigma  programs  across  the  company. 

Analysis 

Data  provided  by  Company  F  suggests  that  they  have  a  solid  financial  plan  and  footing.  They  are 
conservative  with  regard  to  venturing  into  areas  of  potential  growth  as  a  result  of  a  former  company 
culture  of  too-rapid  expansion  and  subsequent  marginal  financial  performance.  This  is  evident  in  our 
rating  of  Company  F  and  apparent  from  their  qualitative  comments.  They  are  hesitant  about  rapid  growth 
that  could  create  financial  problems,  and  struggle  with  investing  in  new  staff  and  operational  methods 
(lean/six  sigma). 

Company  F  has  a  focus  on  responding  to  their  customers.  Their  prime  customers  are  in-the-field 
technicians,  and  their  equipment  is  designed  to  work  simply  to  provide  equipment  performance  data  that 
can  be  used  without  requiring  operator  analysis.  As  such,  the  company  readily  provides  field  support  and 
warranty  work  so  that  the  technician  customer  has  confidence  and  loyalty  to  their  brand. 

There  was  little  evidence  of  Lean  and/or  six  sigma  processes  in  their  manufacturing  and  business 
operation.  Operating  statistics  were  not  evident,  and  admittedly,  they  perform  to  ISO  9000  rather  than 
developing  a  Lean  operation.  Supporting  this  observation  was  management’s  statement  that  the  one  thing 
they  would  change  to  improve  performance  was  implementation  of  mature  processes  across  the  company. 

Summary  of  Results 

Figure  27  provides  a  comparison  of  all  data  from  the  RIPA  survey  of  Company  F.  This  graphic  suggests: 

•  There  is  agreement  on  financial  viability  between  the  consensus  RIPA  scores  of  the  UT  team  and 
27  item  input  method  of  the  management  team  on  the  present  state. 

•  There  is  agreement  between  the  9  item  present  and  past  states  and  the  discussion  from 
management  about  the  company’s  past  to  present  performance.  Both  methods  indicate  better 
financial  performance  in  the  present.  Improved  leadership  is  credited  with  the  current  stability  of 
the  company,  as  suggested  in  the  management  debrief  and  evidenced  in  the  past  and  present 
bubbles. 

•  All  scores  suggest  below  average  operational  processes.  This  was  supported  by  both  the  UT  team 
observations  and  discussions/RIPA  scoring  by  management. 

Company  F  was  the  first  company  visited  and  served  as  a  pilot  for  the  assessment  process.  No  interviews 
with  employees  were  done  in  this  interview.  One  of  the  assessors  had  prior  knowledge  of  the  company 
and  provided  insights  to  the  other  team  members.  After  determining  the  value  of  this  extra  information, 
the  employee  interviews  were  added  as  a  mandatory  component  of  the  assessment  process. 
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APPENDIX  J:  COMPANY  G  VISIT  SUMMARY 


Introduction 

Company  G  provides  specialty  materials  as  a  military  supplier.  They  have  a  limited  market  with  no 
presently  identified  commercial  applications  for  their  product.  Their  facility  has  a  full  range  of  special 
testing  services  for  dynamic  and  environmental  conditions. 

Company  G  is  running  at  only  50  percent  utilization  and  has  excellent  R&D  capability  to  support  their 
product.  They  are  presently  ramping  up  from  a  research  house  to  a  product  supplier  of  their  proprietary 
technology.  They  estimate  16  percent  of  the  market  share  in  their  product. 

Company  G  is  moving  from  a  pure  research  facility  to  research  and  production.  They  are  presently 
growing  their  manufacturing  capability.  They  intend  to  move  from  50  percent  research/50  percent 
production  to  20  percent  research/80  percent  production  within  three  years. 

Company  G  is  ISO  9001  and  AS9004  certified  and  have  an  excellent  quality  assurance  program. 

Interview  and  Tour  Summary 

All  employees  from  the  top  management  to  the  shop  floor  workers  were  excited  to  talk  with  us  about  their 
product  and  their  company.  They  have  a  family  atmosphere  and  take  pride  in  their  support  of  military 
operations.  They  have  an  adequate  facility  in  size,  tools  and  supporting  equipment.  They  have  recently 
invested  in  new  machinery  that  is  intended  to  take  them  from  a  research  house  to  a  manufacturing  facility. 

The  senior  management  have  been  with  the  company  for  many  years  with  the  exception  of  a  newly  hired 
manufacturing  manager  who  is  a  Six  Sigma  Black  Belt.  He  will  also  act  as  the  Quality  Assurance 
manager  and  appears  to  be  starting  to  implement  Six  Sigma  tools  to  track  their  process.  They  have  used  a 
communication  and  progress  tracking  system  rather  than  process  control. 

Company  G  has  excellent  product  display  so  each  worker  knows  how  important  each  of  their  individual 
jobs  are.  The  shop  was  neat,  but  did  not  appear  to  have  a  good  process  flow.  Their  shop  has  a  good 
emphasis  on  safety  and  workers  stated  they  really  enjoy  working  there,  it’s  like  a  family. 

Management  Debrief 

The  management  team  was  interviewed  regarding  1)  how  they  have  changed  in  the  last  five  years  and  2) 
what  they  believe  to  be  their  biggest  challenges  and  obstacles. 

Methodology 

The  Company  G  management  team  evaluated  their  company  in  four  different  ways.  Each  data  set  was 
provided  independently  by  each  member  of  the  Company  team  in  paper  and  pencil  mode.  The  four  data 
sets  are: 


•  Data  Index  6:  Scoring  of  the  27  items  for  present  state. 

•  Data  Index  7:  Scoring  of  the  company  on  the  nine  categories  in  its  present  state  (1  through  5 

scale). 

•  Data  Index  8:  Scoring  of  the  company  on  the  27  items  five  years  ago. 

•  Data  Index  9:  Scoring  of  the  company  on  the  nine  categories  five  years  ago  (one  through  five 

scale). 

The  management  team  did  not  discuss  their  ratings  or  reach  consensus. 

The  data  was  input  into  the  RIPA  assessment  spreadsheet  by  the  UT  team  of  researchers. 
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The  UT  team  arrived  at  their  RIPA  results  by  each  team  member  independently  answering  the  27  items 
(data  index  3)  and  then  reaching  consensus  as  a  team  on  the  final  score  for  each  of  the  nine  categories 
(data  index  5). 

Data 


RIPA  Data 

Figure  28  provides  the  nine  item  scores  as  generated  by  the  RIPA  calculator  for  the  Company  G 
management  team’s  input  of  the  27  items.  Figure  29  provides  the  UT  team’s  scores  for  the  nine  items  as 
generated  by  the  RIPA  calculator  of  the  27  items  and  the  consensus  scores. 


Figure  28.  Company  G  -  Management’s  Rating  From  Input  of  27  Items  (Data  Index  10) 
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Figure  29.  Company  G  -  UT  Team’s  Evaluation  From  Consensus  (Data  Index  5) 


Figure  30  provides  a  graphical  comparison  of  the  management  team’s  evaluation  of  now  and  five  years 
ago,  and  the  evaluation  from  the  UT  team. 

Company  G  managers  evaluated  their  company  using  the  27  items  both  now  and  five  years  prior.  The 
data  used  in  the  Company  G  management  charts  comes  directly  from  their  scores  on  the  27  items. 
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Company  G 


Figure  30.  Company  G  -  All  RIPA  Evaluations 
(Company  G  Now,  Company  G  Five  Years  Ago  and  UT  Team) 

Qualitative  Data 

The  management  team  responded  to  two  questions  to  provide  qualitative  data.  The  intent  of  this 
questioning  was  to  gather  supporting  information  for  1)  the  five  year  previous  RIPA  input  and  2)  the 
present  RIPA  from  a  casual  dialogue  with  the  Company  G  management  team. 

What  has  changed  over  the  last  five  years? 

•  Wanted  new  leadership  in  the  company  and  we  went  and  found  it. 

•  In  the  last  18  months  we  have  started  putting  process  controls  in  place. 

•  Customer  has  forced  us  to  mature  and  move  from  research  facility  to  also  manufacturing. 
What  challenges/obstacles  face  your  company? 

•  Need  for  everyone  to  understand  our  new  culture  about  what  is  acceptable. 

•  Everyone  has  to  learn  about  process  control  -  know  what  they  need  to  do  and  then  do  it. 

•  Be  consistent  in  management. 


Summary  of  Results 

•  From  Figure  28  and  Figure  29,  Company  G’s  highest  scores  were  in  Market  Analysis.  The  UT 
team  rated  them  the  lowest  in  Fean  Operations  as  they  were  just  starting  to  implement  Fean 
techniques.  Quality  is  the  second  lowest  rating  from  the  UT  team. 
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•  From  Figure  28,  Management  rated  themselves  the  lowest  in  Profit  and  Cash  Flow,  which  is 
interesting  considering  they  would  not  share  financial  data  with  the  UT  group.  Management’s 
third  lowest  rating  was  Quality,  which  trends  with  the  UT  rating. 

•  Company  G  stated  that  they  had  hired  new  managers  in  the  last  two  years  to  help  make  the  shift 
to  a  quality,  production  facility  from  a  research  oriented  facility.  This  was  evident  in  their  lower 
score  in  all  categories  from  five  years  ago. 

•  Company  G  management  rated  themselves  high  in  Marketing  due  to  their  recent  work  in 
preparing  a  marketing/business  plan.  This  plan  was  not  rated  quite  as  highly  by  the  UT  team,  and 
contributed  to  the  lower  Operational  Excellence  score  by  the  UT  team  in  Figure  30. 
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APPENDIX  K:  WEEK  3  -  RIPA  FIELD  WORK,  JULY  30  -  AUG  3,  2007 

As  a  part  of  the  validation  process  and  on-going  training,  a  one  week  training  class  was  conducted  for 
AFRL/RXM  ManTech  personnel.  The  class  agenda  is  provided  as  electronic  attachment.  The  course 
evaluations  were  very  positive  and  are  also  provided. 

Some  key  facts  about  the  class  were: 

a.  The  purpose  for  the  training: 

•  To  provide  a  review  for  prior  participants  whom  had  not  participated  in  a  RIPA  event  since  their 
training. 

•  To  provide  practice  in  the  RIPA  assessment  for  AFRL/RXM  ManTech  personnel  who  had  either 
a  condensed  version  of  the  course  or  no  prior  training.  In  each  case,  the  participants  had  not 
previously  performed  a  RIPA  assessment. 

•  To  provide  data  to  be  used  in  the  validation  process. 

The  learning  objectives  were  as  follows; 

•  Utilize  the  RIPA  as  a  tool  to  understand  the  stability  of  the  company,  its  progress  with  lean,  the 
strengths  and  weaknesses  that  may  affect  the  probability  of  success  for  the  technology  transition 
(Module  III). 

•  Understand  what  type  of  assessment  team  should  be  assembled  to  do  the  RIPA,  financial  analysis, 
and  any  other  activities  deemed  necessary  (Module  III). 

The  weekly  schedule  is  provided  an  electronic  attachment. 

b.  Potential  user  groups  who  would  need  this  type  of  training  are  engineers  and  program  managers  with 
the  need  to  participate  in  the  assessment  of  the  operational,  financial  and  supply  chain  readiness  of  a 
company.  Participants  will  have  had  little  prior  training  and  experience  in  the  topics. 

c.  Methods  and  sources  used  to  develop  the  training.  This  course  was  developed  for  AFRL/RXM 
ManTech  was  accomplished  under  a  previous  contract. 
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Course  Feedback 


University  of  Tennessee 
“Industrial  Preparedness  RIPA  Practice” 

Wright  Patterson  Air  Force  Base 
July  30-  August  3,  2007 

1.  Did  the  course  content  meet  your  expectations?  Please  explain. 

Yes,  I  was  expecting  to  get  hands  on  practical  knowledge  &  experience.  Provided  more  practice.  I  felt  a 
sense  of  participation  with  having  participated  in  earlier  training.  For  me  a  RIPA  refresher. 

I  don’t  have  any  experience  in  this  area  and  the  course  gave  me  some  fundamental  skills  and  a  tool  to 
complete  a  valid  assessment  with  confidence  in  the  results. 

Yes,  I  was  told  the  course  would  help  us  to  better  assess  a  plants  manufacturing  ability.  Wasn’t  sure  what 
to  expect  but  was  a  good  review  &  practice  of  the  process  of  RIPA. 

I  didn’t  have  any  previous  training,  but  I  still  got  a  lot  out  of  the  course.  For  the  most  part,  the  material 
was  well  organized.  I  was  looking  for  the  opportunity  to  practice  the  walk  through’s  &  pulling 
information  out  of  the  companies  representatives. 

No. .  .1  have  taken  the  class  before  so  there  was  nothing  new.  I  took  the  3wk  already.  Good  info¬ 
refresher. 


2.  Please  comment  on  the  application  of  the  course  content  to  your  organization. 

Will  help  me  provide  a  better  understanding  of  RIPA  and  MRA’s  to  my  leadership.  We  will  conduct 
RIPA  on  all  contracts. 

This  will  help  on  issues  &  developing  proactive  action  plan.  As  we  practice  the  RIPA  concepts,  I  will 
apply  it  more  to  MRA  performed  on  Title  III  and  Mantech  Projects. 

Very  applicable.  Regularly  engage  with  new,  smaller  -  size  companies  and  need  to  assess  their  condition. 
Assist  me  in  a  rapid  assessment  of  the  general  ability  of  a  company  to  implement/manufacture  a  new 
technology  or  end  item. 

I  feel  it  will  be  able  to  help  us  on  our  MRA’s.  We  will  use  it  in  our  MRA  process.  The  RIPA  tour 
questions,  evaluation  criteria  seemed  more  in  depth  than  I  would  expect  from  an  MRA. 

The  content  is  very  valuable  for  performing  MRA’s. 

Very  applicable  since  our  personnel  need  to  be  able  to  perform  manufacturing  assessments.  Will  help 
teaching  RIPA/MRA. 

I  believe  there  is  a  disconnect  between  RIPA  and  our  MRA  process.  Frankly,  I  think  RIPA  is  most 
applicable  to  Title  III  programs  where  I  am  concerned  more  about  the  business  aspects  than 
manufacturing. 

I  think  this  content  is  “spot-on”  for  meeting  needs  of  our  org.  Great  course! 

Directly  applicable  to  MRL  activities  and  strengthens  our  core  competencies  of  industrial  readiness  & 
manufacturing  readiness.  I  will  use  it  as  I  get  involved  in  “Mantech  MRA’s”. 

Needs  more  “follow  on”  focus.  For  instance,  why  not  bring  in  a  “user”  of  the  RIPA  tool,  who  has  used  it 
to  make  a  decision  for  selecting  one  supplier  out  of  many. 
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3.  Were  the  on-site  visits  helpful? 

17  Yes  _  No  1  Somewhat 

Please  explain: 

It  helps  to  gain  experience  in  asking  questions. 

I  gained  the  most  benefit  from  applying  what  I  learned  with  the  experience  of  conducting  the  RIPA’s. 
Extremely  helpful. 

Chance  to  apply  what  is  taught.  Provided  good  spectrum  to  benchmark. 

Helped  me  to  learn  what  to  look  for  and  how  to  apply  the  knowledge  gained  from  the  course.  So  many 
different  types  of  manufacturing  methods,  plant  layouts,  and  quality  programs. 

The  practice  was  very  helpful. 

In  order  to  make  an  assessment,  you  need  to  interact  with  company  personnel.  The  course  would  be 
almost  worthless  without  the  site  visits/Emphasis  on  multiple  visits  to  hone  RIPA  skills!  Site  visits  are  a 
“must”. 

They  all  need  to  be  Vi?  hr  to  45  min  in  conference  room  and  then  lhr  or  less  tour. 

Visual  examination  of  how  vastly  different  manufacturing  processes  employ  lean  for  profitability  and 
operational  excellence  was  enlightening. 

4.  How  did  the  RIPA  material  add  to  the  course? 

Material  helped  to  conduct  RIPA.  Just  right  amount  of  material  in  the  notebook. 

New  RIPA  tools  very  good  addition.  Streamlining  tool  use  good. 

We  need  a  more  “consolidated”  handbook  as  a  reference  tool. 

Vital  to  understanding/utilizing  the  tools.  Helped  with  our  assessments. 

Interesting,  but  somewhat  confusing  when  trying  to  think  of  an  MRA.  Especially  the  ratio  information. 

Provided  Industrial  preparedness  assessment  training.  It  was  good  to  go  through  financial  definitions 
again. 

Perfect.  Gave  me  the  calculations  and  formulas  to  apply. 

Sufficient  and  useful. 

5.  Please  rate  the  overall  service  of  the  Center  for  Executive  Education  including  coordination  and 
customer  support. 

_ 11_  Excellent  7  Good  _  Fair  _  Poor 

Comments: 

Good  coordination  &  customer  support. 

The  instructors  encourage  interaction  within  the  team  structure. 
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6.  Please  rate  AFIT  conference  room  experience. 


5  Excellent  12  Good  2  Fair  Poor 


7.  Please  rate  your  satisfaction  with  the  faculty.  Please  circle  your  choice. 

(l=low;  5=high) 


Instructor 

Instructor’s  contribution  to 
what  was  learned  in  class 

Instructor’s  preparation, 
organization,  and  ability  to 
communicate  ideas 

Instructor’s  use  of 
class  time 

Alan  Pannell 

4.65 

4.76 

4.71 

Elaine  Seat 

4.41 

4.71 

4.59 

9.  Would  you  recommend  this  program  to  a  colleague? 

17  YES  1  NO 


10.  Additional  Comments: 

More  control  over  tours  and  suppliers  visits. 

Kudos  for  working  to  keep  our  folks  focused  on  asking  correct  questions  -  breaking  out  of  their 
propensity  to  drill  deep  on  technical  issues. 


63 


